ibm java batch solutions - ibm www page · java on z/os jvm launcher solutions websphere java batch...

Unit 1a - Overview

Unit 1a - 1

© 2013 IBM CorporationIBM Advanced Technical Skills

IBM System z and z/OS

Java Batch SolutionsVersion: September 10, 2013

IBM System z and z/OS

Java Batch SolutionsVersion: September 10, 2013

IBM z/OS Java Batch Solutions

2

© 2013 IBM CorporationIBM Americas Advanced Technical Skills

Gaithersburg, MD2

Version: September 10, 2013

This page intentionally left blank


3


Gaithersburg, MD3


Agenda / Table of Contents

Material Available Online …

Overview

Java on z/OS

JVM Launcher Solutions

WebSphere Java Batch

Java Batch in CICS, DB2

Batch and Decision Management

Open Standard Java Batch

This is the agenda for the session.


4


Gaithersburg, MD4


This Presentation Available Online

Overview …

ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP101783

PDF in Chart FormatShowing the charts as you're seeing them here

PDF in Speaker Notes FormatSame charts, but printed so the speaker notes associated with each chart appear.

A quick reminder that the presentation is available online under "WP101783" on Techdocs.

Two formats are provided -- chart format and speaker note format. The speaker notes provide a further

explanation of the key points being made on each chart.


5


Gaithersburg, MD5


OverviewSetting the stage for the discussion of modern batch solutions


6


Gaithersburg, MD6


Batch (or Bulk) Processing

Examples of bulk processing …

Many definitions exist ... they all have in common the relative lack of human interaction, and the expectation of results at a future time rather than immediately:

Request

Response

Request

Response

In general:

● Interaction is one-for-one ... that is, request with matching response

● Expectation is for response to follow request in a near-immediate span time frame

Online Processing

Submit

Complete

Start

EndProcess

In general:

● Interaction is one-for-many... that is, initial request results in many results from processing

● Expectation is for results to finish within some determined non-immediate time frame

Bulk Processing

We start with a compare-and-contrast between online processing and the focus of this presentation, batch

processing.

The point of contrast we wish to focus on is the interaction between the request and the processing that takes

place. Online processing has, in general, a one-for-one relationship. In addition, there is an expectation of the

response being returned in a timely fashion.

Batch processing on the other hand has, in general, a one-for-many relationship between the request and the

result. The request initiates some background processing against a set of data. The result is the completion of

the processing against the data. The expectation is that the processing will be completed in some timely but non-

immediate fashion.

This is why we introduce the term "bulk" to this discussion. The focus is on processing bulk data in some form.


7


Gaithersburg, MD7


Some Examples of Batch (or Bulk) Processing

What's Behind This? …

Just to set some context for the upcoming discussion

Month-End Tax or Fee Calculation and BillingCustomer records are processed with tax or other calculations processed against activity. This may be part of a larger process of calculating amount owed and formulating billing.

Period-End Statements and ReportsAn example is investment portfolio results and reporting.

Data TransformationRaw data records are processed with goal of transforming some aspect of the data content or layout. Result is a second set of data with the intended formatting.

Data AnalysisData records are analyzed to determine trends or patterns. Data mining to find new potential markets is one example. Analyzing vast quantities of seismic data is another.

Point is these are activities where processing in bulk is a better way to maximize efficiency of data access and computer resources. The

completion time is determined but not immediate.

Here we offer a few examples of batch processing.

Note: conduct a short discussion of batch processing examples seen in the environment of the presentation

audience. That will allow the discussion that follows to relate to the specific examples.

Ultimately the point of bulk processing is that it allows more efficient use of people and data processing equipment. The nature of the processing does not demand an immediate response (though, as the chart

indicates, the completion time does need to understood and predictable).

Batch processing exists because the nature of the work suggests doing the work in bulk. It's one part of the

broader spectrum of overall information processing.


8


Gaithersburg, MD8


What's Behind This?

Approaches to Batch Modernization …

Every business has different motivators. The common ones we've seen:

Batch Window CompressionThe window in time for batch processing is shrinking. There's a need to better manage online and batch processing concurrently within the same system.

Java Skills and Common ToolingJava skills are more common that traditional programming skills. Further, Java tooling for online work is powerful and capable of being used for batch programming as well.

GP cycles and Offload to Specialty EnginesCost pressures are creating a need to explore ways of offloading processing cycles from GP to specialty engines such as zAAP or zAAP-on-zIIP.

Other motivators may exist. The key is that these motivators are real and they are driving exploration of modern batch.

The movement towards Java batch on the mainframe has several components to it. The points made on this

chart represent some of the more common things we hear.

Note: A discussion with the presentation audience on the business motivators behind their investigation of Java

batch is recommended. There may be business drivers not outlined here. That will allow better discussion of the

relevance of the technology to the business needs as the presentation unfolds.

The three issues outlined on the chart are:

● Batch Window -- in the past the "batch window" was a dedicated time during which batch processing was

done and online processing was suspended. But those days are largely behind us -- online is 24 x 7 and in

general batch must be done concurrent with online.

Note: with proper capacity planning and system goal management it's quite possible operate COBOL batch concurrent

with online processing. So there's nothing magical about Java that allow concurrent processing. The point here is that

Java batch processing within the same runtime framework as Java online processing facilitates concurrency a bit better.

That's one of the key value statements of WebSphere Java Batch.

● Skills and Tooling -- COBOL is an excellent programming language, and very good development tooling

exists for it. But in today's world the prevalence of Java skills exceeds COBOL skills, and the trend continues in that direction. Further, many have very large investments in Java development tooling and the desire is to

leverage that tooling investment into the world of batch development as well.

● CPU Offload -- batch executing in COBOL runs on general processors, and general processor cycles are

seen as having a much higher cost profile than System z specialty engines such as zAAP, or zAAP-on-zIIP.

So the motivation here is to reduce the cost profile of batch processing by moving some batch to Java.

Note: the question of what batch processing to consider a candidate for Java batch is an important discussion

to have. Many factors play into the decision -- Java for new batch vs. rewrite existing; and if rewrite, then the

complexity and inter-dependencies of the existing. The complexity / inter-dependency question gets to the

cost and risk profile of a rewrite project. In general, when starting out, tackling complex batch that's on the

critical path is not recommended.


9


Gaithersburg, MD9


Approaches to "Batch Modernization"

Framework for discussion …

Generally speaking, we see two basic approaches:

Preserve Existing ... Java Batch for NewLeave existing batch processes as they are today, but as new requirements come up then engineer them into the Java batch model

Re-Engineer Some Existing Batch ProcessesIdentify existing non-Java as candidates and then re-engineer them to operate in a Java batch environment

Typical starting approach: identify batch processes with fewer interdependencies and then work out from there

What we don't see is a "rip-and-replace" strategy. That's too costly and too risky.

Always a reasoned incremental approach.

Others who have understaken the "batch modernization" exercise have tended to approach this in one of two

general ways:

1. Java for new batch -- that is, fence off existing batch processes and preserve them as they're operating

today, but make a point to engineer any new batch processes using Java.

2. Incremental Re-engineering -- which implies identifying candidates for rewrite to Java and then undertaking

a software development project to perform the re-write and test of the new code.

With respect to the second approach, we find most approach this with a reasonable degree of caution: the initial

candidates are typically batch processes with relative few interdependencies, and batch processes off the batch

critical path. When skills and experience are gained, then other batch processes are worked on.


10


Gaithersburg, MD10


Framework for Discussion That Follows

Java on z/OS …

The following diagram illustrates the points we will touch on as we review the modern batch technologies available on System z and z/OS:

Modern Batch Technology

What it isAn explanation of its intended purpose

How it worksAn explanation of its functions and features

DifferentiatorsA discussion of how the technology compares to others discussed, and the particular value this technology offers

How it takes advantage of System zAn explanation of System z and z/OS exploitation that may occur with the technology

How it integrates with existing processesAn exploration of integration and usage patterns when other components in your overall architecture are involved

This chart provides an understanding of the discussion framework to follow.

Note: this presentation was structured on the premise that what is desired by the audience is an understanding of

the technology and help with the positioning considerations between technologies. That's what allows for an

informed decision. This presentation is designed to be a reasoned walk-through of the technology with points

along the way to discuss the potential business value.


11


Gaithersburg, MD11


Java on z/OSA story of taking advantage of the platform


12


Gaithersburg, MD12


IBM Java on z/OS

Specialty Engines …

It's important to understand that while the Java APIs are industry standards, the implementation below the APIs becomes increasingly platform-aware:

ProgrammingAPIs

High-Level JVM Functions

Low-Level JVM Functions

SDK conforms to the accepted standards● IBM SDK provides all the required APIs according to the

specification at the level being discussed● IBM z/OS SDK provides additional APIs to take advantage

of z/OS platform specific functions (such as SAF security)

JVM Functions common across IBM SDKs● The JVM is entirely IBM's ... first delivered in 2005● Many features: generational GC, shared classes● High-level JVM functions common across IBM Java

System z and z/OS functions● Takes specific advantage of platform, including exploitation

of new CISC instructions available with new System z: z10, z196, EC12

● Big Decimal, Large Page, Out of Order execution, transactional execution, flash paging ... equals performance

● Work with z/OS dispatcher to offload to specialty engines

There's a slight tension in the Java story that needs to be understood before going into technical detail -- that is,

from an application perspective Java is common across all platforms, but below that application interface line the

Java implementation is very much platform aware.

So three broad points are made on this chart:

1. IBM Java, regardless of platform, is Java ... it complies fully with the Java specification.

2. The implementation of the Java Virtual Machine (JVM) under the application specification level is fully IBM's design and development, based on IBM's many years of experience in operating system and virtual OS

environments.

3. IBM Java on z/OS is platform aware and takes advantage of specific platform features to provide either

additional function or improved performance.

The first point is important because it establishes IBM Java as being specification compliant ... applications are written to the particular Java specification level, not to some IBM version of Java. Java is Java at the API spec

level.

The second point is important because it establishes the JVM implementation as being something IBM has

designed, developed and continues to improve. Provided the application specification interfaces are compliant

(which they are), the underlying implementation may be done however a vendor chooses. IBM's JVM is based on

IBM's design.

The third point is important because helps to address the question of "Why run Java on z/OS?" The answer is

because (a) it performs very well, due in part to the specific exploitation of System z and z/OS features Java on

z/OS does, and (b) it offloads to specialty engines.

The rest of this section expands on these key points.


13


Gaithersburg, MD13


System z Specialty Engines

Timeline …

Specialty engines provide additional processing capacity with an attractive financial profile: lower acquisition cost, not counted towards software license charges

Java Work1 DDF and DB2 Utilities2 Other Work

z/OSDispatcher

zAAP Processors zIIP Processors General Processors

zAAP - System z Application Assist ProcessorOffload of Java and XML parsing work.

zIIP - System z Integrated Information ProcessorCertain DB2 work and XML parsing services.

Note 2 -- Plus other work, see http://www.ibm.com/systems/z/hardware/features/ziip/

Note 1 -- See http://www.ibm.com/systems/z/hardware/features/zaap/

zAAP-on-zIIPA means of more efficiently using specialty engines by defining only a pool of zIIP processors and allowing eligible zAAP work to run on the zIIPs3.

zAAP-on-zIIP

Note 3 -- EC12 planned to be the last system that supports zAAP; after that, zAAP-on-zIIP will be the offload mechanism

Note: simplified representation of

offload mechanism. Various tuning and control parameters exist.

IFL - Integrated Facility for LinuxFor running z/VM and Linux. Does not apply to z/OS, but plays strong role in Linux for System z

We start this discussion with a high-level explanation of System z "specialty engines" for those who may not have

a background in the topic. The ability to offload Java work to specialty engines is one of the key motivators to

using Java for batch on System z.

Specialty engines provide a means of dispatching work running on z/OS to engines other than the general

processor engines. That provides two key benefits:

● It allows work that must run on general processors (such as CICS, or COBOL batch) to have capacity on

those engines by not using GP cycles for Java, and

● It allows Java work to run on engines that do not count towards software licensing charges.

The net effect is that adding Java work to a z/OS system does not necessarily imply having to acquire addition GP

engines, which then implies potential increases in software licensing charges. It is possible that Java work can

have a cost-neutral effect (aside from the acquisition cost of the specialty engines, which are priced substantially

below GP engines).

Note: the dispatching of Java work to available specialty engines is transparent to the Java applications. The

decision where to dispatch is done at a level lower on the system ... down at the interface between the JVM and

the z/OS dispatching code.

There are several flavors of specialty engines and those are discussed on the chart above. With respect to Java

the key is zAAP, or zAAP-on-zIIP. (The EC12 machine is the last to support zAAP alone; going forward it will be

zAAP-on-zIIP, which is a means of consolidating non-GP work on a pool of specialty engines rather than having to

separately allocate zAAP and zIIP engines. In short, zAAP-on-zIIP is a good thing.)

In summary, the existence of specialty engines on z/OS is a key enabler for Java work. It is a factor often cited by

customers as a factor that plays a role in their decision to host Java work on the mainframe.


14


Gaithersburg, MD14


IBM System z Java Product Timeline

Performance …

This shows the level of commitment to Java and Java on System z:

SDK 1.1.1● First OS/390 Java product● Followed by 1.1.4 and 1.1.6● 31-bit

SDK 1.1.8● 31-bit

SDK 1.3.1● 31-bit● z/OS ● Linux for System z

SDK 1.4● 31-bit z/OS● 64-bit Linux for System z

SDK 1.4.2● 64-bit z/OS● First use of IBM J9 VM

and JIT Technology

SDK 5● 31-bit and 64-bit● z/OS and Linux for System z● z9 exploitation

SDK 6.0.0● Java SE 6 APIs● 31-bit and 64-bit● z/OS and Linux for System z● z10 exploitation

SDK 6.0.1● Java SE 6 APIs● 31-bit and 64-bit● z/OS and Linux for System z● z196 exploitation● Enhanced JZOS and z/OS security

SDK 7● Java SE 7 APIs● 31-bit and 64-bit● z/OS and Linux for System z● More z196 exploitation● zEC12 exploitation● OpenJDK

1998

1999

2000

2001

2002

2003

2004

2005

2006

2007

2008

2009

2010

2011

2012

This is a busy chart, but don't let the busy nature of the chart distract you. The chart is telling a very good story.

What this chart represents is the time line of improvements and enhancements to the Java on System z product

since its beginning back in late 1997. From that point forward it is a fairly steady set of improvements ...

improvements in specification level supported and improvements in platform exploitation.

Note: this chart is included to establish the important point that IBM invests in the development of Java on the

System z platform. Java on System z is designed to be fully aware of the platform on which it runs and take

advantage of features and functions found there. Ultimately that translates into performance, as we'll see on the

next chart.


15


Gaithersburg, MD15


IBM z/OS Java Performance

Summary …

The story is one of improvement in hardware and Java software

zEC12 SDK 7 SR3

zEC12 SDK 7 SR1

z196 SDK 7 SR1z196 SDK 6 SR8

z10 SDK 6 SR4

z10 SDK 6 GM

z9 SDK 5 SR5

CPU Intensive Multi-threaded Workload on 16-way Machine

Better scaling slope leading up to CPU

over-commit

Around 12X improvement

CPU intensive workload so scaling

past over-commit not expected

Four things going on: (1) faster hardware, (2) exploitation of machine instructions, (3) exploitation of machine features (Flash Express paging with large pages), and (4) increasing efficiency of the JVM function itself

Disclaimer: controlled environment with

specific conditions during measured testing. Results may vary. This is not a promise of performance or results.

-Xaggressive and large page

This picture comes from another presentation related to Java performance on z/OS. The chart is showing the

performance improvements that have been achieved from the starting test point (IBM Java SDK 5 on a System z

z9 machine) to the most recent (SDK7 on zEC12 machine). The test design charts the performance for the

workload from 1 processing thread to 32 on a 16-way machine.

Notes: the various lines can make this chart a bit confusing at first. The main point we're making with this chart is

the performance improvement seen over time as the SDK levels have improved and the machine hardware has

improved. That's the two blue circles connected with the dashed line.

The workload for this test was very CPU-intensive, which meant that a dispatched thread would consume a CPU

pretty much exclusively while it ran. That's why the scaling goes up from 1 to 16, but then tapers off from 16 to 32

threads. We focus on the 16-thread data points because of the nature of this test workload.

Performance has improved over time ... 12 times from lower data point to higher data point. That's due to the four

factors outlined on the chart. This bears out the implied message of the previous chart -- that IBM has invested in

the SDK on z/OS, and the effect of that manifests as improved performance.


16


Gaithersburg, MD16


Section Summary

JVM Launchers …

A very broad summary of the Java on z/OS section:

IBM Java is fully compliant with Java specificationsProgramming APIs compliant with the specification

IBM Java on z/OS has programming extensionsProvide you option to code apps to extensions for added functionality

IBM Java has IBM JVM, GC and JIT TechnologyBased on decades of experience in operating systems

IBM Java on z/OS takes advantage of System zWhich manifests as performance improvements over time

IBM continues to invest in Java on z/OSCommitment to the platform is evident

We summarize this section with the points made on this chart.

Note: the objective of this section was to establish the viability of Java on System z and z/OS, and to overcome

objections that may have been based on experiences from many years ago. IBM Java on z/OS in 2013 is not like

Java on z/OS from 1999. Considerable progress has occurred in the years since initial release.


17


Gaithersburg, MD17


JVM Launcher SolutionsBPXBATCH, JZOS and the Batch Runtime


18


Gaithersburg, MD18


What is a "JVM Launcher?"

Three JVM Launchers on z/OS …

It's simply a means of initiating a JVM start and the running of Java programs in it:

Four questions arise:

1. What JVM launcher technologies are available on z/OS?

2. What does each provide in addition to the JVM?

3. Why would one be used over another?

4. If not JVM launcher, what else might be considered?

Java Program

Java Virtual Machine

Instantiation JVM tear-downProgram Execution

Launch the JVM

In this section we will discuss JVM Launcher technologies. JVM Launchers are designed to instantiate a JVM and

run a program, then take the JVM down after the Java program has completed. In this section we will discuss

three such technologies available on z/OS.

Note: later we will discuss an alternative approach, which is to use a JVM that stays active after the Java work

completes. That approach avoids the overhead of instantiating the JVM and tearing it down time and time again.

In this section we will seek to answer the four main questions shown on the chart (along with any other questions

that may arise during discussion).


19


Gaithersburg, MD19


Three JVM Launcher Technologies We'll Explore

BPXBATCH …

For the purposes of this discussion we'll focus on three:

BPXBATCHProvides a UNIX shell environment to JCL. From the UNIX shell you may then issue a "java" command to launch the JVM and run the specified program.For purposes of comparison, consider this equivalent to invoking "java" from a shell script or manually from a command line.

JZOSProvides a more comprehensive JVM launcher utility that overcomes some of the shortcomings of BPXBATCH. Includes a set of class libraries and JNI native to access z/OS functions and services.

z/OS Batch RuntimeProvides a functional subset of the WebSphere Java Batch environment for running Java batch programs. Includes such things as transactional management, integration with COBOL, and transactional integrity between Java, COBOL and DB2.

These are the three JVM Launcher technologies. All three are functions included with the z/OS operating system.

The primary difference between the three is the Java batch functionality each provides. The BPXBATCH solution

provides essentially none beyond what the JVM itself provides. JZOS expands on that with a toolkit for access to

z/OS services above and beyond what a simple JVM provides. The Batch Runtime provide more still -- a runtime

in which Java can interact with COBOL and share a JDBC Type 2 connection with transactional coordination.


20


Gaithersburg, MD20


BPXBATCH (and BPXBATSL)

JZOS …

Provides access to a UNIX shell environment from which you may instantiate a JVM and run a program. It works:

http://publib.boulder.ibm.com/infocenter/zos/v1r13/topic/com.ibm.zos.r13.bpxa500/bpxbatr.htm

//BPXBATCH JOB ...

//STEP EXEC PGM=BPXBATCH,

// PARM='SH java HelloWorld'

//STDIN DD DUMMY

//STDPARM DD DUMMY

//STDERR DD SYSOUT=*

//STDOUT DD SYSOUT=*

//STDENV DD *

CLASSPATH=/u/myclasses

/*

//*

JVM instantiated from shell with java command, with Java class file specified.

Parameters to Java program may be specified trailing class name in PARM=' ' or in location specified by STDPARM DD.

If STDIN DD specified, it must be a z/OS UNIX file.

STDERR and STDOUT may be JES spool, or UNIX file, or MVS data set

BPXBATSL is like BPXBATCH, but it allows child processes to be fork/exec'd within the same address space. (Restrictions apply, so see InfoCenter article below.)

It provides a JVM with some basic access to z/OS DD statements. It has limitations with respect to other z/OS services. That's why

JZOS was developed and IBM provides that with z/OS.

The first technology is BPXBATCH, which is a function provided with the UNIX Systems Services (USS) function

of z/OS. It is designed to provide a UNIX shell environment to JCL batch. That then provides you the ability to

launch the JVM and pass in a parameter that names the Java class file to execute.

It works, but it doesn't provide much in the way of additional services to your Java program other than access to

UNIX files for input and output. That means if your Java batch program seeks to do more sophisticated batch

processing -- such as transactional management and checkpointing -- that would the responsibility of your Java

code. That implies writing custom code above and beyond the business logic of your batch processing. The

desire to avoid such custom middeware code is precisely why IBM offers other Java batch solutions.

BPXBATSL provides essentially the same thing as BPXBATCH but avoids multiple address spaces when child

processes are forked.

Summary: BPXBATCH works, but nobody seriously advocates it as the solution for enterprise Java batch

processing. It's acceptable for small-scale ad hoc Java invocations but is not robust enough for larger-scale

processing. It's included in this presentation mostly to put a stake in the ground for the low end of the Java batch

processing scale.


21


Gaithersburg, MD21


Overview of JZOS

JZOS Toolkit …

JZOS is technology originally developed by Dovetail Technologies and acquired by IBM. It is now supplied as part of z/OS.

http://www.ibm.com/systems/z/os/zos/tools/java/products/jzos/overview.html

JVM Launcher

Toolkit

JZOS

Launch a JVM and run a Java program● One address space; all accounting data attributes to the job or started task● Mix Java and non-Java steps in same JCL● Pass condition codes between Java steps, or between Java and non-Java● Access MVS data set via DD statements in Java

Toolkit provides Java classes and native libraries● Methods for allocating, deleting, and renaming MVS data sets● Interface for reading and writing sequential data sets in record mode● Methods for issuing single and multi-line WTOs● Interface for invoking DFSORT and directing I/O into/from Java applications.● Java APIs for submitting z/OS batch jobs from Java● Java APIs that provide interface for invoking IDCAMS (VSAM)● Java APIs for many z/OS functions:

Obtaining job/step/user names, process ids, etc. Writing SMF records Reading environment variables

A more flexible JVM launcher mechanism along with a set of class and native libraries which provide Java programs access to z/OS services

Because of the limitations of BPXBATCH for running Java, the people at Dovetail Technologies developed the

JZOS function. IBM has since acquired the technology and now supplies it with the z/OS operating system. See

the URL at the bottom of the chart for the IBM web page that discusses JZOS.

JZOS provides two key things as part of its function:

1. A JVM launcher that overcomes some of the shortcomings of BPXBATCH

2. A set of Java class libraries and native z/OS libraries to provide program API access to z/OS services

Note: JZOS having two functional components -- improved JVM launcher and service access toolkit -- is not

immediately obvious. Please do not assume the audience is aware of this distinction. The distinction is important because the two may be used independently from one another:

● The JVM launcher component may be used even if none of the toolkit services are needed by Java program

you are running. So JZOS may be meaningfully used for simpler Java batch programs.

● The JZOS toolkit is not limited to just JZOS usage. It consists of a set of class libraries and native libraries,

which other Java runtimes may exploit. Keep that in mind when we discuss the Batch Runtime and the WebSphere Java Batch section.

Without delving deep into the details of JZOS at this point, the key point is that JZOS represents an improved means of launching a JVM and running a Java program. It is far more flexible in its design than BPXBATCH, and

provides a valuable means of mixing Java and non-Java in a multi-step JCL. It's part of z/OS so no additional

licensing is required. It is still a JVM launcher, so if your Java batch requirements call for many repeated

invocations you need to consider whether the overhead of all those JVM create/destroy cycles is worth it, or if you

should go to a model where the JVM stays active (WebSphere Java Batch).


22


Gaithersburg, MD22


JZOS Toolkit Usable Outside JZOS Launcher

JZOS Launcher …

The JZOS JVM launcher and the JZOS toolkit are not tightly coupled. The toolkit services may be used by other Java Batch mechanisms:


z/OS Batch Runtime

Toolkit Classpath and Libpath updates

z/OS Services Offered by JZOS Toolkit

We've not yet covered WebSphere Java Batch or z/OS Batch Runtime

Key point is JZOS toolkit and those offerings are not mutually exclusiveAnd as we'll see, the z/OS Batch Runtime in fact uses the JZOS JVM launcher as well

This chart illustrates the point we made in the previous chart's speaker notes -- that is, the JZOS toolkit consists of

a set of Java class libraries and z/OS native libraries that may be used by other Java runtime environments.

The z/OS services made available by the JZOS toolkit may be of great value to your Java batch plans. But you

may need to use one of the other runtimes. That's okay ... they work together.

Consider the JZOS toolkit as a set of utility services that runs with JZOS and other Java runtimes.


23


Gaithersburg, MD23


z/OS JZOS JVM Launcher

Multi-Step Jobs …

It comes with the z/OS SDK installation. Sample JCL is provided along with the JZOS load module. Invocation involves customizing sample JCL:

/usr/lpp/java/J7.0/mvstools

JVMLDM70

/samples

/jcl

JVMJCL70

JVMPRC70

Note: JZOS is specific to the Java version and the bit-mode. Here SDK 7 31-bit is shown.

● Load module -- copy out to LIBRARY data set

● Calling JCL -- this JCL calls the proc and passes variables

● Procedure -- this receives variables and invokes JZOS

JCLJCLNames the Java class to invoke

Sets up the environment variables

Provides STDIN DD (optional)

Passes arguments (optional)

PRCPRCReceives Java class and arguments

Calls JZOS load module and specifies the Java class file and any passed-in arguments

Defines standard DD statements

Starts address space, instantiates the JVM,

loads the class, invokes the main()

method and supplies any arguments

This is chart that shows how JZOS is delivered. The key is that it's supplied as part of the z/OS Java SDK and it

comes with sample JCL you customize for your environment.

Note: this is standard z/OS JCL system programmer structure. There's nothing mysterious about it at all. JZOS

is supplied as a load module that is copied out to a LIBRARY data set, and JCL in two forms: JCL for the

submitted job, and a JCL procedure that is called from the job JCL. Your specific Java job information is coded in

the job JCL, and that is passed over to the proc, which is generic and receives the customized values as passed

arguments from the job JCL.

It's also worth noting that the JZOS load module is specific to the SDK level and bit-mode in use. So if you have

need for both 31-bit and 64-bit JVMs, then you would copy out the load modules and JCL from the 31-bit and 64-

bit directories of the z/OS Java SDK you wish to use.

The JCL launches the JZOS address space and instantiates the JVM within it. Then it loads the Java class you

named and invokes the main() method. When your Java completes the JVM is torn down and the address

space is shut down.


24


Gaithersburg, MD24


JZOS and Multi-Step Batch Jobs

Batch Runtime Overview …

JZOS allows you to mix Java and non-Java as steps in a larger batch job. Condition codes set by each step available to other steps in the job.

//MYBATCH JOB ...

//STEP1 EXEC PGM=COBOL1

: (step DD cards)

//STEP2 EXEC PROC=JVMPRC70,

// JAVACLS='com.ibm.MyClass1'

: (step DD cards)

//STEP3 EXEC PGM=ASSEMBLER

: (step DD cards)

//STEP2 EXEC PROC=JVMPRC70,

// JAVACLS='com.ibm.MyClass2'

: (step DD cards)

One job

Multiple steps

Mixed programs

One address space

Same WLM policyService Class (goals)Reporting Class (statistics)

Accounting data consolidated under one job

COBOLCOBOL

JavaJava

AssemblerAssembler

JavaJava

The purpose of this chart is to show how JZOS can be used in cooperation with non-Java in a multi-step JCL

batch job. JZOS provides the ability to do this with the attributes shown on the right side of the chart. You could

code BPXBATCH on a step, but it would run in a separate address space, operate under different WLM policies,

and be accounted for separately.

So JZOS represents a step forward from BPXBATCH for invocation of Java main() program execution. In

general JZOS is preferred over BPXBATCH for Java execution. (BPXBATCH is still very good for general UNIX

shell work; JZOS is for Java only ... it can't be used to invoke a shell script for example.)

Is JZOS the final answer? Maybe. Maybe not. It depends on what your Java batch processing requirements are.

For example, while JZOS provides some really nice z/OS access services, it does not provide much along the way

of batch functions such as transaction coordination and checkpoint processing. The next function discussed -- the

Batch Runtime -- does.


25


Gaithersburg, MD25


z/OS Batch Runtime Overview

z/OS 2.1 enhancements …

This is function supplied with z/OS 1.13 and higher. It provides a means of integrating Java and COBOL with DB2 JDBC T2 connection shared and TX coordination:

Java Program

DB2 z/OS

COBOL Program● Start TX● Call DB2● Invoke COBOL● Finish TX

● Business Logic● Call DB2● Business Logic● Return

Batch Runtime Transaction and Execution Services

RRS Shared Attach

z/OS 1.13 Batch Runtime

JCLJCL

Submit

Or initialize with COBOL and have it call Java ... it works in either case

The other JVM Launcher technologies provide nothing in the way of TX services. If your code

has TX requirements, then you write custom code.

With the z/OS Batch Runtime the TX services provided and help coordinate between Java and COBOL when using JDBC Type 2 to local DB2.

z/OS 2.1 adds PL/I

The z/OS Batch Runtime is function provided starting with z/OS 1.13 that takes JZOS a bit further by providing a

batch processing framework inside the JVM that is launched by JZOS. That batch processing framework is a

subset of that provided in the WebSphere Java Batch function we'll discuss next.

The key batch assist function provided by this Batch Runtime is the ability to cooperatively process Java and

COBOL as part of a larger batch process, and to have a JDBC Type 2 connection shared between the two with

coordinated transaction processing.

Note: pause and think about how much custom code you would need to write to perform just the functions

mentioned in that last sentence. The code to invoke COBOL from Java implies JNI function, which is not trivial to

write. The shared JDBC Type 2 connection implies writing to the RRSAF interface. The code to create and

manage a transactional context is also not trivial, particularly if proper transaction recovery is taken into account.

This is the value of the Batch Runtime -- it provides batch container services so your Java code can stay focused

on the batch business logic you wish t perform.

The Batch Runtime may be used even if you don't call COBOL, or don't need shared JDBC Type 2 with

transaction coordination. Those are the key features we highlight to show you the power of it.

We will offer a positioning chart in a few moments that will position BPXBATCH against JZOS and the Batch

Runtime function. All three are JVM launcher technologies. They operate with different levels of provided

function, and the programming model for the Batch Runtime is different from the programming model for JZOS

and BPXBATCH.


26


Gaithersburg, MD26


z/OS 2.1 - Enhancements to Batch Runtime

JZOS and z/OS Batch Runtime …

The Batch Runtime with z/OS 2.1 comes with enhancements:

z/OS 1.13 z/OS 2.1

Java main() Programs

Single Step Jobs

COBOL

DB2 and JDBC T2

Java main() Programs

Single Step and Multi-Step Jobs

WebSphere Java Batch programsExceptions:● Java SE runtime, not Java EE● No WAS functionality (EJB, MDB, JCA)● No WLM classification, no SMF 120.9● Parallel Job Manager not available

COBOL

PL/I

DB2 and JDBC T2

Transactional VSAM

More on WebSphere Java Batch coming up

The Batch Runtime planned for z/OS 2.1 adds several enhancements, including support for PL/I and Transaction

VSAM. In addition, it will add the Java batch container function from the WebSphere Java Batch included with

WAS V8.5. So some of the container services provided will be available to Java programs in this runtime, and

Java batch applications written for WebSphere Java Batch are compatible with this Batch Runtime environment.

Note: the key differentiation between this Batch Runtime and the WebSphere Java Batch function is the Java

runtime under the batch container. In WebSphere Java Batch it's WebSphere Application Server with its full Java

EE programming model; with the Batch Runtime the WAS runtime is not present and the JVM supports Java SE,

not Java EE. We'll cover the WebSphere Java Batch function in detail later in this session.


27


Gaithersburg, MD27


JZOS and z/OS Batch Runtime

Integration with Enterprise Schedulers …

The two are not mutually exclusive. In fact, the Batch Runtime makes use of JZOS to launch the JVM and load the Batch Runtime function:

com.ibm.zos.batch.container.BCDBatchContainer

BCDPROC

BCDBATCH

JZOSLauncher

JZOSLauncher

z/OS Address Space

31-Bit Java Virtual Machine

z/OS Batch Runtime

JavaCOBOL

JZOSToolkit

JZOSToolkit

z/OS Servicesz/OS Services

General Requirements:● IBM 31-bit SDK for z/OS, V6.0.1

Must be 31-bit because COBOL is a 31-bit language

● IBM Enterprise COBOL Version 4.2

● DB2 V9 with PTF UK62190 for JDBC 3.0 specification, or PTF UK62191 for JDBC 4.0 specification

● DB2 V10 with PTF UK62141 for JDBC 3.0 specification, or PTF UK62145 for JDBC 4.0 specification level

Calling JCL / procedure similar to JZOS; Batch Container

supplies samples of its own

The JZOS toolkit may also be used in conjunction with the

Batch Container

The Batch Runtime is a JVM Launcher solution, and in fact makes use of JZOS as the JVM launcher. So the

runtime construction is similar to JZOS in that there's a job JCL and a proc JCL. The difference is the Java class

that's called and loaded by JZOS is the IBM-written Java class that instantiates the Batch Runtime function in the

JVM launched by JZOS. The Java batch job that is then run is supplied in z/OS 1.13 via inline JCL control

statements that pass into the container the batch job to run.

The Batch Runtime has a set of general requirements as shown on the chart above.


28


Gaithersburg, MD28


Integration with Enterprise Schedulers

JVM Launcher Positioning …

All three JVM launcher technologies discussed may be initiated via submission of JCL. That becomes the integration point with enterprise schedulers:

Enterprise Scheduler Function

Job JCL

Java Program

Java Virtual Machine

JES SpoolOr elsewhere if JCL defines other than spool

Submit

In many ways this is business as usual ... to the Enterprise Scheduler function the submitted JCL job is the batch job. The invocation of a JVM

to run Java is a detail not of concern to the scheduler function

This is the first time in this presentation we've raised the issue of integration with Enterprise Scheduler functions

such as IBM Tivoli Workload Scheduler or other comparable vendor products. We didn't mention it with

BPXBATCH or JZOS because those were so clearly invoked with JCL, and the integration point with the

Enterprise Scheduler was at that point of JCL submission.

The same is true with the Batch Runtime. That makes sense when you step back and realize the Batch Runtime

is really an augmented instance of a JVM launched by JZOS. The Batch Runtime in fact relies on JZOS to be the

JVM launcher. So the point of integration with Enterprise Schedulers is the same as for BPXBATCH or JZOS -- at

the point of JCL submission.

This is true even if the Batch Runtime is simply one step in a larger multi-step batch job submitted by the

Enterprise Scheduler. That's one of the values of using JZOS as the JVM launcher is that it allows for coordinated

Java and non-Java steps within a multi-step JCL. To the Enterprise Scheduler the JCL that it submits is the batch

job.

Note: the reason we're bringing up the Enterprise Scheduler integration is because (a) eventually it becomes a

point of interest in most discussions about Java batch processing, and (b) it becomes an important point later

when we discuss WebSphere Java Batch. So we introduce the topic here where the integration is relatively

obvious, and we build on it with WebSphere Java Batch where the integration involves a supplied utility as you'll

see later.


29


Gaithersburg, MD29


JVM Launcher Positioning

Summary …

The following provides a high-level positioning of the alternatives discussed so far:

JVM Bit-mode

Batch Services

DB2 Access

TX Coordination

Programming Model

BPXBATCH JZOS Batch Runtime

Address Spaces

31-bit or 64-bit 31-bit or 64-bit 31-bit

Multiple(one with BPXBATSL)

One One

NoneProvides a JVM but no

other extra-JVM services

SomeToolkit provides z/OS

services. Batch services such as checkpoint not provided

Yesz/OS 2.1 - WebSphere Java

Batch batch services (checkpoint, data stream, etc.)

JDBC T2 or T4Access code is all

custom to the application

JDBC T2 or T4Access code is all

custom to the application

JDBC T2 onlyTX coordination requires RRSAF connect to DB2

z/OS 2.1 - tVSAM as well

NoneAny TX must be custom

implementation

NoneAny TX must be custom

implementation

YesProvided by Batch

Runtime and coordination with RRS

Java main()Standalone Java

Java main()Standalone Java

WebSphere Java Batchz/OS 2.1 - Based on subset of WebSphere Java Batch model; runs in J2SE not J2EE, and no

WAS classes available.

Here is a positioning chart showing the three JVM launcher technologies across the top with a set of functional

considerations down the left side of the chart.

Note: the functional considerations are by no means exhaustive. If other considerations come up during

presentation of this deck, please forward an e-mail to [email protected] and let me know. I can expand

the positioning to a second chart and continue the discussion.

BPXBATCH and JZOS appear to be very similar if you look at just this chart, but as we discussed earlier JZOS

provides a number of improvements over BPXBATCH for Java processing. The Batch Runtime offers some more

distinct differences from JZOS because a batch container is provided, and container code to allow COBOL to run

in the same address space as Java is provided. The programming model also changes: BPXBATCH and JZOS

are simple Java main() programs, whereas the Batch Runtime complies with the WebSphere Java Batch

programming model we'll discuss in the next section.

Summary: all three are JVM launcher technologies, but the Batch Runtime is what provides a Java batch

container with additional batch services. In z/OS 2.1 the batch services in the Batch Runtime are similar to the

services provided in WebSphere Java Batch. The big difference there is that WebSphere Java Batch is build on

WAS, which is a runtime model where the JVMs stay active. That alleviates the overhead of JVM instantiation and

tear-down, which may play a key factor when daily batch processing involves hundreds or thousands of such batch

cycles.


30


Gaithersburg, MD30


Section Summary

WebSphere Java Batch …

A very broad summary of the JVM Launcher section:

JVM Launchers instantiate JVMs and run programGenerally an "application per JVM" model

We showed three JVM Launcher ModelsThey range from simple (BPXBATCH) to more functional (Batch Runtime)

Which you use depends on your business needsBPXBATCH may work fine for simple ad hoc functions; JZOS provides a more

flexible launching model with z/OS access services; Batch Runtime addresses the need for Java/COBOL interaction with coordinated transaction support.

Potential downside: cost of JVM start and stopOne potential downside of JVM launchers is the overhead associated with starting and stopping the JVM. May not be an issue if cycles are few; tends to become an

issue when the number of start/stop cycles increases.

This is the summary chart for the section just completed.


31


Gaithersburg, MD31


WebSphere Java BatchJava Batch inside WebSphere Application Server


32


Gaithersburg, MD32


WebSphere Java Batch Overview

Two Delivery Mechanisms …

The following is a very high-level overview of the WebSphere Java Batch function:

WebSphere AppServer Instance

EJB Container

Web Container

Batch Container

Java Batch Applications

IBM RAD Tooling Support

Dispatcher Function

z/OS Operating System

System z HW Platform

Job Submission Interfaces

xJCLxJCL

Job Properties Declaration File

Key Points:● This JVM stays up for as long as the WAS server stays up

Days or more ... avoids overhead of JVM instantiation in environments where batch invocation is high

● All the services of WAS z/OS are available to the batch applications running thereTransaction, data access, Parallel Sysplex and clustering, WLM, SAF, SMF

● The batch container provides valuable batch services that keep batch logic focused on the business and away from writing custom middleware codeMulti-step jobs, Batch Data Stream Framework, Checkpoint, Skip record, Step retry, Parallel Job Manager

TechDoc: ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP101783

● Browser● Command Line● SOAP web services● RMI● MDB

The next function to discuss is WebSphere Java Batch. This is function that started out as a separate licensed

product called "Compute Grid," and has been incorporated into the WebSphere Application Server product starting

in WAS V8.5.

The first important point to stress about this solution is that it's built on the runtime model of WebSphere

Application Server, which means the JVMs are long-lived. What the WebSphere Java Batch function provides is a

batch container environment that operates alongside the Web and EJB containers of WAS.

That is a different execution model from the JVM Launcher technologies, where the JVM was instantiated and a

named program run. Here the JVM is already up, so another mechanism is needed to provide an interface to tell

the batch container to invoke a deployed Java batch program. WebSphere Java Batch provides a "dispatcher

function" for that purpose.

Finally, the ability to layer a batch container on top a Java EE runtime allows for some very interesting batch

services to be made available to your programs. That's what we'll cover in this section.

Note: The WP101783 Techdoc at the URL shown on the lower-left of the chart contains a set of documents

related to WebSphere Java Batch.


33


Gaithersburg, MD33


At Present Two Delivery Mechanisms

Container Services …

The "WebSphere Java Batch" function is delivered in two ways:

WebSphere Application Server

Version 7 or Version 8

WebSphere Application Server

Version 8.5

Compute Grid V8WebSphere Java Batch Product

Add to WAS ("Augment") WebSphere Java BatchFunctionally equal to CG V8

For those who have a standardized commitment to

WAS V7 or V8 but desire WebSphere Java Batch function

WebSphere Java Batch function provided in WAS V8.5 product

Note how both are functionally identical

At this point in time there's a bit of confusion about how to acquire the WebSphere Java batch function. That's

because there are two avenues -- one as a separately licensed product called Compute Grid, the other as part of

WebSphere Application Server V8.5.

The separate Compute Grid product is available for those who have WAS V7 or V8 (but not V8.5) and are not in a

position to migrate their application runtime up to WAS V8.5. The Compute Grid product may be added (the term

used is "augmented") to the WAS V7 or V8 runtime to provide the Java Batch function to WAS.

Starting with V8.5 the Compute Grid function was incorporated into the WAS V8.5 runtime foundation. With V8.5

there is no need to acquire or augment Compute Grid. It is included with the product.

Note: the key point is that Compute Grid V8 and WAS V8.5 represent the same Java Batch functionality. Which

you choose is more a matter of your runtime version considerations rather than functional differences.


34


Gaithersburg, MD34


Overview of Batch Container Services

Job Submission and the JMC …

The following provides a summary. The WP101783 Techdoc has many more details:

Batch job dispatching and management system● Separates job invocation and management from the

instantiation of the JVM● Provides job control: stop, restart, cancel, suspend● Multiple interfaces: Web, SOAP, RMI, command, MDB

Data record read and write support services● Abstracts low-level stream handling from business logic● Framework passes data to batch logic as Java object

Checkpoint and job restart services● Checkpoint tracking abstracted from batch logic

and performed as function of container● Commit and rollback a function of the container● Job restart from last checkpoint

Job resiliency services● Ability to skip and track records where data read

or write operation throws exception● Ability to retry job steps when unhandled

exception is thrown● Ability to utilize defined call-out points in batch

job execution to invoke custom Listener code

Parallel job management and execution services● Develop and deploy applications to run segments in

parallel based on your designation of data partitioning● Ability to programmatically partition data ranges based on

passed-in parameters● Container manages top-job and all sub-jobs to completion

COBOL Container Support● Invoke COBOL from Java● JNI stub code provided with container● Call stub generator utility to create call stubs and data

parameter code

Do you recall back during the discussion of BPXBATCH and JZOS we said that neither provided much in terms of

batch services? We indicated with those JVM launchers the responsibility would be yours to code any such

services as custom code.

The purpose of WebSphere Java Batch is to provide a container-managed environment that provides batch

services to your Java batch job. This chart provides a bullet-point summary of many of those services.

Note: the key point to stress here is that these functions are provided as programmatic interfaces to your batch

function. Your batch function calls these functions as needed, using the appropriate documented API. A review of

the bullet points on the chart will show this is a fair amount of function provided as container services, which

highlights the value of WebSphere Java Batch as a Java batch runtime.

There's not enough room in this speaker note section to provide details on every one of the bullets. The

WP101783 Techdoc at ibm.com/support/techdocs provides two-page brochures that describe many of the

functions shown above.

When presenting this chart some balance needs to be achieved -- some time spent is necessary to make the point

about the services provided, but covering in detail each bullet point is probably not necessary. A recommended

approach for discussion would be:

● Start with checkpoint services and make clear that the batch job step code handles none of this. This is all container-

managed with transaction management being provided by WAS and checkpoint intervals a configurable element of the

job declaration file.

● Point out that job resiliency functions are to let long batch jobs continue when the occasional glitch happens. Better to let

glitch pass and log problem than hold up whole job because of a small hiccup.

● Discuss at a high level the concept of partitioning data ranges so multiple batch jobs may run in parallel, each executing

against its data range. The service provided here is (a) programmatic interfaces to allow job partitioning at time of

submission, and (b) a coordination layer that tracks the sub-jobs as a logical entity to completion.

● Refer to the COBOL container as something very similar to what the Batch Runtime function provided, but with some

differences to be covered a few charts later from this chart.


35


Gaithersburg, MD35


Job Submission and Monitoring

CI Model …

The Job Management Console (JMC) is the browser interface. It allows you to submit and monitor the jobs:

Job State Flow

Jobs go through a series of states in this model. The interface to the

Dispatcher Function provides you control to submit, view, cancel, stop,

suspend, restart or remove jobs

Before going into any details, let's first see an example of the Job Management Console showing several jobs that

have been submitted. This affords an opportunity to illustrate what the "Dispatcher Function" is doing and the

"states" a submitted job goes through.

The Job Management Console (JMC) is the browser interface to the Dispatcher Function of WebSphere Java

Batch. It is one of several interfaces; the others being command line, SOAP web services, RMI and MDB. The

role of the Dispatcher Function is to provide a means of submitting jobs and controlling them. We show the

browser interface here simply because it's the one most likely used first, and it's the easiest to approach

conceptually.

A "job" consists of two key components: the Java code that implements the batch processing and a job declaration

file that tells the system how to process the job. The Java code is packaged in a standard EAR file format and is

deployed into WAS like any other EAR file. The job declaration file is called "xJCL" and it is read by the

Dispatcher Function to begin execution and control the operational behavior during execution. We'll see more on

the xJCL in a few charts.

Jobs go through "states" in this process. A typical series would be "Submitted" � "Executing" � "Ended." If

something is wrong the job may go into an "Execution Failed" state where you the batch operator would then read

the job log to see what happened and take corrective action. Once a job begins execution you may wish to cancel

or stop the job, where it will then go into a restartable state. You may then restart the job and it will pick up and

continue.

Note: We have not yet introduced the concepts of "compute intensive" (CI) or "transactional batch" so the

explanation of the difference between Cancel/Stop and "Suspend" might not be entirely clear. That said,

Cancel/Stop is for transactional batch and Suspend is for CI jobs. In a transactional batch job "Cancel" will cease operation and rollback to the last checkpoint; "Stop" will allow execution up to the next checkpoint and then stop.

The point of this chart is to begin the discussion with a brief illustration of the job interface structure. Anyone in the audience with a background in starting jobs using z/OS JES should relate to this.


36


Gaithersburg, MD36


High-Level: "Compute Intensive" Model

Transactional Batch Model …

This provides a simple execution model suitable for (a) computational work, or (b) a means of wrappering existing Java main() programs to run:

:

public class MyCIWork implements com.ibm.websphere.ci.CIWork {

:

public void run() {

Your batch code goes here

}

}

Batch Controller BeanStateless session bean provided by container framework

Success if no Exceptions

Thrown

Dispatch

Very simple model to develop and run

Very few batch runtime services available● No data read or write patterns to utilize ... roll your own● No checkpoint processing ... roll your own● No commit and rollback ... roll your own

Use tooling to wrapper Java main() into this framework and it will run in WebSphere Java Batch

Useful model for some types of applications.

Transactional Model may be better suited to your needs ...

Now we begin a high-level discussion of the programming model used by WebSphere Java Batch and, it should

be noted, the Batch Runtime function discussed earlier (since it is based on WebSphere Java Batch).

We're going to start with a programming model that is the most simple. It's called the "Compute Intensive" model,

or CI for short. The CI model provides a basic single-step framework in which a Java program may run.

Note: it is important to set some context before describing this CI model and why we start with it. It's likely most

customer Java batch programs using WebSphere Java Batch will use the transactional batch model described

next. That's the more intuitive "read a record, process the record, write the record" model.

We start with this CI model because (a) it's really simple, and (b) it's the "on-ramp" into WebSphere Java Batch for

people who have an asset base of Java main() programs they wish to run in WebSphere Java Batch. Point (b)

is the key reason why we start with this -- because saying that a Java main() can be wrappered in this CI model

and run within WebSphere Java Batch makes intuitive sense. That allows people with an inventory of Java

main() programs to understand they can easily participate in WebSphere Java Batch if they choose.

Originally the CI model was intended for just as the name implies -- computationally intensive batch programs,

such as you might find in mathematical or scientific data processing. There data I/O is not as important as

climbing on the CPU and crunching numbers for an extended period of time. Hence the name, and hence the

relatively simple one-step structure.

But as it turned out, this model was also very good for wrappering Java main() programs and running them

under the job submission and job management structure of WebSphere Java Batch. So it represents a means of

enjoying some of the benefits of the container-managed model of WebSphere Java Batch. And, as noted, it

provides an intuitively approachable way of preserving existing Java main() assets.

Note: the picture shows a "Batch Controller Bean," which then invokes the run() method. The batch controller

bean is provided as part of the packaging of the deployable EAR file. It is what the job dispatching function

described earlier interacts with to submit and manage jobs within the WebSphere Java Batch model. Consider it

part of the WebSphere Java Batch infrastructure. It's exposed here to "demystify" how things work in this model.


37


Gaithersburg, MD37


High-Level: "Transactional Batch" Model

xJCL …

This is what people usually think of when they think of "batch." At the heart of this is the job step code, which encapsulates the data record business logic:


Dispatch

:

public class MyJobStep implements BatchJobStepInterface {

:

public void createJobStep() {

}

public int destroyJobStep() {

}

public int processJobStep() {

Step logic for a data record here

if record not null: return BatchConstants.STEP_CONTINUE;

if record null: return BatchConstants.STEP_COMPLETE;

}

}

Controller bean invokes job step in iterative loop until STEP_COMPLETE returned

Your job step logic would process data record

according to business needs

Now we introduce the "transactional batch" programming model, which is built around the batch step which

handles one record at a time. The batch step processJobStep() method is invoked iteratively by the controller

bean until the step returns "complete." As long as "continue" is returned the controller bean knows to turn and

invoke the batch step class again.

Note: questions may come up at this point -- (a) multi-step jobs allowed? Answer: yes, discussed next chart; (b)

where does batch step get data record? Answer: either from custom code in batch step or by calling batch data

stream framework implementation, discussed in an upcoming chart.

The key here is that the looping logic is not part code you provide in the batch step. The batch step operates

against a data record and performs whatever business logic against the data record that is necessary. The

iterative looping continues until the record retrieval results in no more records, at which time the batch step returns

"complete." That signals to the batch controller to then invoke destroyJobStep() and the step completes.

As noted, the batch step implementation should not handle looping counts; it should not to to manage transactions

or do commits or rollbacks. Those are all functions externalized to the container and controlled by the job

declaration file (xJCL) which we'll describe next.

Note: it is useful to step back and view this conceptually. The transactional model is a data-record oriented

process, and the batch step class focuses on a data record. Think about what kind of processing against a data

record your batch might need -- update a field, write a new record based on data read in, etc. That's the very core

of your transactional batch processing; the rest is related to managing the iterative loop and all that surrounds that

-- starting the looping, knowing when looping is complete, maintaining the checkpoint count, performing commits

and rollbacks as needed. That's where the batch container of WebSphere Java Batch comes into play. That also

why a job declaration file is so important: it provides the container an understanding of how to manage your "job," which contains your "step." So we look at xJCL next.


38


Gaithersburg, MD38


Job Declaration File - xJCL

Checkpoint processing …

Conceptually, xJCL is just like normal // JCL -- it describes the job to be run and the context in which the job is to operate. The difference is xJCL is written in XML:

<?xml version="1.0" encoding="UTF-8" ?>

<job name="MyJob" ... ">

<substitution-props>

<prop name="inputDataStream" value="/tmp/input-text.txt" />

<prop name="outputDataStream" value="/tmp/output-text.txt" />

<prop name="checkPoint" value="10" />

</substitution-props>

<job-step name="MyStep1">

<classname>com.ibm.ws.batch.MyStep1</classname>

<batch-data-streams>

<bds>

<logical-name>inputStream</logical-name>

<props>

<prop name="PATTERN_IMPL_CLASS" value="com.ibm.websphere.batch.EchoReader" />

<prop name="FILENAME" value="${inputDataStream}" />

</props>

</bds>

</batch-data-streams>

</job-step>

:

<job-step ...>

</job-step>

</job>

Note: this is a trimmed version of actual xJCL

Comparable to the JCL "JOB" card. It sets job-level information along with some substitution properties

Substitution properties may be overridden at submission time.

Comparable to the JCL step. It names the Java class that

implements the step function.

The "Batch Data Stream" implementation is specified. In this

example it defines the class that implements the input stream.

If job consists of more steps they are specified in sequence.

This tells WebSphere Java Batch what to run and how to run it

This is a topic that can't be covered in detail on one chart. So the objective will be to establish a conceptual

framework of what xJCL is and how it's used.

Note: if the audience has z/OS people in attendance, then it is useful to start by pointing out that this is very similar

to normal "slash-slash" JCL ( // ) used over the years, except this is written in XML. That provides a way of

relating this new thing (xJCL) to what they already know (JCL), and moves the learning down to a matter of

understanding syntax rather than understanding concept.

At the very highest level, the xJCL file is used to tell the WebSphere Java Batch runtime that a job invocation is

being requested, and to provide the runtime an understanding of what job to run and to provide details about the

job. (That is exactly the same high-level purpose of regular JCL.)

Keep in mind that an objective of WebSphere Java Batch is to keep Java batch step logic focused on the business

needs. Therefore, one of the roles of WebSphere Java Batch and xJCL is to externalize details that are resolved

at time of job submission. For example, details about the location of a data input file should not be hard-coded in

the business logic. Those are externalized with symbolic references. Again, the JCL DDNAME concept is a

useful way of relating xJCL to normal JCL. The same concepts apply.

Note: it is also useful to remind people of two things about the creation of xJCL -- (1) like regular JCL, a common

practice is to take an existing file and modify it as needed. Almost nobody creates regular JCL from scratch. And

(2) the tooling support for WebSphere Java Batch provided in IBM Rational Application Developer provides an

"xJCL Wizard" to assist in the creation of this file.

The structure of xJCL breaks out like normal JCL -- a "job" section at the top, and "step" sections for as many

steps as your job requires. Properties that apply to the whole job are coded up in the job section; properties that

apply to the step are supplied in the step section.

The xJCL is a necessary part of job submission -- you supply your xJCL to the Job Dispatcher function through

one of the interfaces (web being the easiest to use when first starting out). The Job Dispatcher reads in the xJCL

and from that comes to understand what you're asking for. It then gets to work by signaling to the server where

the batch job is deployed to start up and process.


39


Gaithersburg, MD39


Data Record

Data Record

Data Record

Data Record

Data Record

Data Record

Data Record

Data Record

Data Record

Data Record

Data Record

Data Record

Data Record

Data Record

Data Record

Data Record

Data Record

Data Record

Checkpoint Processing

Batch data stream framework …

Checkpoint commit and rollback is a function of the container. It relies on the transaction manager function of WAS itself for this:

Java Batch Application

Batch Container

xJCL where checkpoint declared

WAS Transaction Manager

Commit

Commit

Last good checkpoint value persisted

Checkpoint interval (record or time) specified in the xJCL

This is a function of the batch container, not your application code

As checkpoint intervals are reached, container commits and records the checkpoint attained

In the event of a failure, job may be restarted at the last good checkpoint

Set the checkpoint interval based on your knowledge of balance between recoverability and efficiency

Checkpoint processing handles the commit action against the data resources being updated by the Java batch

application. This function is abstracted away from your batch job and is handled by the batch container.

WebSphere Java Batch is built on top WebSphere Application Server, and WAS has within it a transaction

manager. So this checkpoint processing function supplied by the container is really relying on WAS's TX manager

to handle the creation of the transaction and managing the commits and rollbacks.

Two checkpoint metrics are supported: number of records or time since last commit. In either case, the value for

the checkpoint metric is specified in the xJCL. Your batch application does not get involved with this at all.

WebSphere Java Batch maintains a relational table entry with the last good checkpoint value reached for every

batch job. This persisted checkpoint value is used in cases where a restart is needed. The container reads the

last good checkpoint value and restarts record processing from that point.

What the checkpoint value should be is a a balance between recoverability and efficiency. Set the value really low

(checkpoint = 1) and you achieve very granular recoverability but at the cost of doing a lot of commit processing.

Set the value really high (checkpoint = 1,000,000) and any restarts will imply going back a long way and doing

work over again.

Note: the key point here is that the checkpoint / commit processing is using WAS transaction manager function,

and this is all abstracted away from your batch application code. Your code focuses on the business logic needed

to process the batch record. The checkpoint / commit / rollback is handled by the container based on the

declaration in the xJCL.


40


Gaithersburg, MD40


Job Step Class

High-Level: "Batch Data Stream Framework"

Skip Record …

This provides a way of abstracting the data read and write logic away from the batch step code. Several Batch Data Stream patterns are provided:


Dispatch

Batch Data Stream ClassLogic to read or write a record of

data to the data store

Data passed as Java object to and from the BDS implementation

Version 8.5 InfoCenter: search on cgrid_btchpttrnover

Supplied "patterns" for:● Read and write using JDBC

● Read and write JDBC and stored procedures

● Read and write byte data from file

● Read and write text file

● Read and write to z/OS data set

● Read and write using JPA

Options:● Take existing pattern and

customize to your needs

● Create your own custom pattern

● Not use BDS but rather code data record read and write handling as part of batch job step

An implied assumption in our discussion earlier of the batch job step was that the data retrieval came from

somewhere. WebSphere Java Batch provides something called "Batch Data Streams" (BDS) as a means of

externalizing this from the job step. The BDS implementations for a job step are specified as properties in the

xJCL. The container invokes the BDS implementation to get the data record, which is formatted into a Java object

and passed to the job step when invoked.

Several Batch Data Stream "patterns" are supplied with WebSphere Java Batch. They are shown on the chart on

the lower left bullet list.

So with this you have three options for data retrieval and storage -- use an existing pattern (customized as needed

to do the specific read or write your batch step requires), create your own BDS pattern, or not use BDS at all and

instead code your data read and write logic in the job step.


41


Gaithersburg, MD41


Skip Record Processing

Multi-step jobs …

Provides a container-managed way of tolerating data read or write errors so the job itself may continue on. Information about data errors may be logged.

Data Record

Data Record

Data Record

Data Record

Data Record

Data Record

Data Record

Data Record

Data Record

Data Record

Data Record

Data Record

Data Record

Data Record

Data Record

Data Record

Data Record

Data Record


Batch Container

Batch Data Stream Framework Code

Exception!

Exception!

xJCL where skip record policy declared

xJCL tells BDSF:

● How many data read or write exceptions to consume

● What exceptions to consider for skiprecord processing

● Alternatively, what exceptions to exclude from skip-record processing

Objective is to provide mechanism to survive the odd data exception rather than stop the whole job

The BDSF implementation code consumes the exception rather than surfacing it up to the batch step code

A "listener" may be called to log information about the data record where exception occurredMore on this coming up

When the skip limit is reached then further exceptions are exposed up to the batch step implementationWhere it might result in job going into a restartable state. Normal checkpoint restart processing would occur.

Technical Flyer at: http://www.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP101783

The "Skip Record" function provides a means of surviving the occasional data read or write operation exception.

This provides a way for a much larger job to keep going rather than go into a restartable state because of some

odd exception.

Note: this is not to say the data record that had the exception is to be ignored. The listener function we'll describe

in a bit provides the ability to log information about the failure. What this provides is a way for a million record

batch job to complete even though one or two data records threw a read or write exception. Absent this the job

would go into a restartable state at the last good checkpoint value and time would pass while operators noticed the

failure and debugged the problem. If batch windows are tight, better to process 999,998 records with 2 failures

than to stop the whole job because of the 2 failures.

The properties for the Skip Record behavior are specified in the xJCL for the "Batch Data Stream Framework"

(BDSF) implementation. This function applies on a per-BDSF basis. The properties in the xJCL provide

information on how many failures to tolerate and what exceptions to consider or exclude from consideration.

What happens if the limit is reached? Then subsequent exceptions are surfaced up to your batch step

implementation. There your code either has to handle the exceptions or, if not, then it will throw the job into a

restartable state. If this happens then the normal checkpoint processing takes place, and the job is restartable

from that last checkpoint interval.

As mentioned, there's a "listener" function that allows this Skip Record process to give control to code you write to

record information about the record that threw an exception. With that information your auditors may then go back

and see what happened and take necessary corrective action.


42


Gaithersburg, MD42


High-Level: Multi-Step Jobs

Retry step …

A job may consist of multiple steps. The job step execution for each is what was shown earlier. Job steps are executed in sequence according to xJCL:


Dispatch

Job Step Class

Job Step Class

Job Step Class

<job ...

<job-step ... >

</job-step>

<job-step ... >

</job-step>

:

<job-step ... >

</job-step>

</job>

xJCL

STEP_COMPLETE

STEP_COMPLETE

STEP_COMPLETE

Job may consist of one to many stepsMay be mixed CI, TX, Parallel

Steps are executed in sequential order according to xJCL

Supplied results algorithm allows for passing step results to next step

Conditional step processing supported

The construction of WebSphere Java Batch jobs begins to look very much like traditional batch with "Jobs" and "Steps". This is the one of

the benefits of a container-managed Java batch environment.

Earlier we noted that a question will likely come up whether multi-step jobs are supported, and we said the answer

was yes. Now that we have the concept of a "job step class file" in place and xJCL understood at the conceptual

level, now we can paint the picture of an xJCL file that describes a multi-step job. It's really as simple as having

multiple sections in the xJCL file that describe each step. A job section is still needed to describe the overall job,

with one to many step sections to follow.

The steps are processed in sequential order according to what's described in the xJCL. The steps may be of all

the same type (all CI, or all transactional), or mixed type (CI, transactional batch, transactional using the Parallel

Job Manager).

Another function provided by the WebSphere Java Batch runtime is the ability to surface the return code of one

step so it's accessible and understood by other steps. Conditional step processing is supported, so a multi-step

job that fails on step 2 can suspend processing of step 3 until the issue is resolved.

Note: it's worth noting here also that jobs have several "states," one of which is "restartable." So if a job is

submitted and starts executing but throws an error, it will go into a restartable state. You correct the error and

then tell the Job Dispatching function to "restart." WebSphere Java Batch checks the job step last processed and

the checkpoint value last set and restarts from that point.


43


Gaithersburg, MD43


Retry Step Processing

Batch Listeners …

Provides a way of retrying a job step in the event of an exception thrown. If successful on retry then the job continues and your processing completes.

Job Step● Java class● Input / Output● Other Properties


Batch Job

xJCL tells Container:● How many step retries may be attempted● What exceptions to consider for retry-step

processing● Alternatively, what exceptions to exclude

from retry-step processing● Whether to process a delay before

attempting a retry of the step

If exception, retry step n

times

This is at a higher level from Skip Record. This is at the "invoke batch step" level. This provides a way to retry the step for exceptions.

Batch container falls back to last good checkpoint and restarts from there

A "retry-step listener" may be called so you can perform custom action upon retry-step processingMore on "batch listeners" coming up

When limit reached, then job goes into restartable state. Normal restart from checkpoint processing.


Provides a way to survive unexpected transient

exceptions through step retry

One level up from skip record processing is "Retry Step Processing." This provides a mechanism to retry a job

step in the event an exception is thrown that would otherwise put the whole job into a restartable state.

Here again we see the declaration for the behavior of this function provided in the job xJCL. This is done on a

step-by-step basis, and the xJCL properties for this indicate how many retries to attempt, what exceptions to

include or exclude from consideration, and whether to include a delay before attempting a retry.

In the event an exception is thrown that results in this retry function taking effect, the batch container will fall back

to the last good checkpoint location and start again. A "listener" may be called to take action on a retry.

Like skip record, when the limit is reached this function stops retrying and allows the normal "restartable job"

function to take over. This is where you as operator would investigate why the step is failing beyond the retry limit

and take corrective action.

And just like skip record, the objective of this is to allow a large, long-running batch job to survive the occasional

hiccup and continue to completion without manual intervention.


44


Gaithersburg, MD44


Batch "Listeners"

Parallel Job Manager …

These are callout points where your custom "listener" code will be called when key events occur. The callouts are managed by the batch container:



Batch Job

Job Listener

Retry Step Listener

Skip Record Listener

Job Listener● Callouts occur:

Start of the job; Start of each step; End of each step; End of job● Register your code to container with property in xJCL● Use this to perform any special setup or cleanup actions at those

points in the lifecycle of a batch job

Retry-Step Listener● Callouts occur:

When the exception is thrown; When the retry is attempted● Register your listener with code in application createJobStep()

method● Use this to take action at these points, such as logging

information about the exception and the point in the processing where it occurred

Skip-Record Listener● Callouts occur:

On skipped read or skipped write operation● Register your listener with code in application createJobStep()

method● Use this to take action at these points, such as logging

information about the exception and the record skipped


Several times now we've mentioned "listeners" that get invoked at certain points of batch processing. Listeners are

implemented with code you write which you "register" to the batch environment so it knows what to call when those

call-out events take place.

There are three listeners we'll focus on -- Job listeners, Retry-Step listeners, and Skip-Record listeners.

● Job listeners -- this is invoked at the start of the job, and the start and end of each batch step, and at the end

of the job. You register your listener class file by coding a property in the xJCL. At those points during batch

job execution the container will call the methods for that lifecycle point. Your code then does what you

determined was appropriate for that point in the job lifecycle ... setup work, cleanup work, logging ... whatever.

● Retry-Step listeners -- this is invoked whenever a job step is retried using the new Retry-Step processing

function. The listener is called when an exception is thrown that triggers Retry-Step processing, and again just

as the step is being retried. You register your listener with a simple line of code in the batch application itself.

● Skip-Record listeners -- this is invoked whenever a record is skipped using the new Skip-Record processing

function. The listener is called when an exception is thrown that triggers Skip-Record processing. Your code

may then log information about the record that has been skipped and the exception that took place. You

register your listener with a simple line of code in the batch application itself.

Listeners provide you a way to insert custom processing logic into the batch job lifecycle at key points during

execution.


45


Gaithersburg, MD45


Parallel Job Manager

COBOL Container …

This provides a mechanism for partitioning batch activity into parallel jobs based on ranges of data you designate:

Job Dispatching

Function

Batch Endpoint

PJM

Parameterizer

Batch Appl

Batch Endpoint

Batch Appl

Batch Endpoint

Batch Appl

Top-Job xJCL

Sub-Job xJCL

Top-job and job number

Sub-jobs ... top-job number and each sub-job number

State of each job

<job ...

<run

instances="multiple"

jvm="multiple">

<prop

name="parallel.jobcount"

value="3" />

</props>

</run>

:

Job step seen as PJM eligible based on "multiple" specified in xJCL

PJM function receives "top job" xJCL and invokes Parameterizer to perform your custom data range segmentation

Job Dispatcher then dispatches "sub jobs" to eligible endpoints for execution. Each is passed the data select parameters generated by parameterizer

Job Dispatcher coordinates execution of top job and sub jobs as logical entity

Not all jobs are well suited for PJM, but those that are may benefit greatly

Parallel either in multiple endpoints, on multiple threads, or both.

Another element of the batch infrastructure provided by WebSphere Java Batch is the Parallel Job Manager, or

PJM for short. The high-level objective of this function is to provide a way to run multiple batch steps in parallel,

each working on its own data range. If the nature of the batch job lends itself to parallel processing (many batch

jobs do), then use of this function has the potential to reduce wall-clock completion time by splitting out and

running in parallel.

Note: you could create completely separate batch jobs, each designed to work on its own static range of data, and

submit them concurrently. That would be a form of "parallel processing." But that's not what the PJM is doing.

The PJM is providing a means of parallel processing such that the data ranges may be programmatically

determined and assigned, the number of parallel processes assigned at time of submission, and the sub-jobs

treated as part of a larger logical "top-job." The PJM is keeping track of the checkpoint processing for each sub-

job and the state of each sub-job. The top-job does not end until all the sub-jobs end. If a sub-job goes into a

restartable state, the top-job is also restartable. Restarting the top-job results in the affected sub-jobs being

restarted so they may run to completion.

The jobs may run in parallel in different JVMs, or within the same JVM but on different threads.

Note: it is useful to step back and imagine trying to provide this level of parallel job management and control with

custom code. It is possible, but likely not a good use of your time to write custom middleware code like this.

Instead, the batch runtime services of WebSphere Java Batch provides this.

As noted, not all batch jobs lend themselves to parallel processing. But those that do can achieve improved job

completion by splitting out and running in parallel. Naturally the data ranges chosen need to be thought out so

data access locks don't occur, and the processing capacity to run multiple jobs concurrently needs to be in place.

But given all that, then the PJM function may prove a valuable part of batch window planning.


46


Gaithersburg, MD46


Java Batch Integration with COBOL

Enterprise Schedulers …

This is similar in concept to the z/OS Batch Runtime, but with some differences. The fundamental objectives is the same: low overhead integration Java and COBOL:


Batch Container

COBOL Container JNI Code

COBOL Container

COBOL Module

PDSE or USS Directory

WAS z/OS Servant Region Address Space

STEPLIB or LIBPATH

Call Stub Generator

IBM RAD

Call Stubs

COBOL Container manifests as separate LE Enclave inside the WAS z/OS servant address spaceKeeps COBOL/Java separate, so environments stay clean

The JNI code for Java/COBOL exchanges providedKey piece of plumbing provided, which relieves you of writing complex JNI

Call Stub Generator builds Java call stubs and data bindingsBased on what's in COBOL source

Provides a very efficient means of calling COBOL modules as part of a Java Batch solution. The JNI code and Call Stub Generator make using this solution relatively simple.

Share JDBC T2 with TXSimilar to what Batch Runtime provides

Another function of WebSphere Java Batch is the COBOL Container, which provides an efficient means of calling

COBOL modules from the Java batch code. Like the comparable function found in the Batch Runtime discussed

earlier, this allows for JDBC Type 2 connections to be shared between Java and COBOL with the transactional

integrity maintained across the two environments.

Note: the key differences between this Java/COBOL integration and that provided by the Batch Runtime is this:

Batch Runtime allows Java to call COBOL or COBOL to call Java, while this COBOL container only operates Java

calling COBOL; and this COBOL container provides a Rational Application Developer based utility to generate the

call stubs and data binding classes to call from Java into COBOL, which makes using this function relatively easy.

What's the value of this? If you have COBOL batch assets that you wish to incorporate into a batch process,

calling COBOL from Java across this interface is far more efficient than other means of invoking COBOL from

Java. The code path is considerably shorter, and that implies less overhead and less latency per call. In batch

processes where perhaps millions of calls are made every little bit of latency avoided tends to reduce the overall

latency experienced.

The ability to share the JDBC Type 2 connection between Java and COBOL with transactional coordination

provided by the WAS runtime provides a much simpler means of managing TX between the two environments.


47


Gaithersburg, MD47


Integration with Enterprise Schedulers

WLM Classification …

One of the interfaces on the Job Dispatcher functions is MDB. This allows a client program to interact with Java Batch and integrate that system with Ent. Schedulers:

Dispatcher Function

MDB Interface

Output

Input

Enterprise Scheduler Function

"WSGRID"Utility

WSGRID JCL

JES SpoolOr elsewhere if JCL defines other than spool

Submit

Exact same Java Batch processing as before

● Messaging provider either MQ or SIBus component of WAS● WSGRID submits job request to dispatcher● Job output from WebSphere Java Batch flows back to WSGRID● WSGRID stays active for duration of Java Batch job

Enterprise Scheduler sees "batch job" as the WSGRID invocation. That stays up for life of Java Batch job. Job output from WebSphere Java batch flows back to WSGRID. Return code from Java Batch flows back to WGRID.

Earlier we introduced the question of integration with existing Enterprise Scheduler functions, and we explained

that for BPXBATCH, JZOS and the Batch Runtime that integration occurs at the submission of the job JCL.

With WebSphere Java Batch the function is different. The point of integration on z/OS is still with the Enterprise

Scheduler submitting JCL, but it also involves a bit of "glue function" between the submitted JCL and the job

dispatching function of WebSphere Java Batch.

Note: the Job Dispatcher function is hosted in a WAS server, and has several interfaces. The one used for this

integration is a Message Drive Bean (MDB) interface. The "glue" between Enterprise Scheduler JCL submission

and the Dispatcher is a supplied program that uses messaging -- either MQ or the built in SIBus of WAS -- to

submit Java batch jobs into WebSphere Java Batch. That "glue" utility is known as "WSGRID."

An important objective to this design was allowing the Enterprise Scheduler to operate in its normal fashion

(meaning without needing to be modified just for WebSphere Java Batch). That was accomplished because to the

Enterprise Scheduler all it sees is the JCL that submits the WSGRID utility job. The WSGRID utility job then

passes information over messaging to the Dispatcher, which dispatches the job into the WebSphere Java Batch

runtime.

Now here's the important point to keep in mind -- the WSGRID job submitted by the Enterprise Scheduler stays

active for the duration of the Java batch job running back in WebSphere Java Batch. That's how the Enterprise

Schedule "knows" that the Java batch job is still running, or when it has completed. The WSGRID job becomes a

"proxy" for the actual Java Batch job running in WebSphere Java Batch.

Another important point to keep in mind -- during the running of the Java batch job in WebSphere Java Batch, the

Java job output is streamed back over messaging to the WSGRID job, which then pipes it to the STDOUT defined

in the WSGRID JCL. That's how Java batch job output is spooled to JES for archival purposes.

If you don't have MQ you may use the version of WGRID that utilizes the Service Integration Bus (SIBus) of

WebSphere Application Server. The same concept of job submission via messaging applies, but the WSGRID

utility program is a Java program, and that Java program bootstraps into the WAS environment and accesses the

built-in messaging of the SIBus. Output is streamed back to STDOUT of the WSGRID job.


48


Gaithersburg, MD48


WLM Classification

SMF 120.9 …

The Dispatcher Function on z/OS has the ability to assign a WLM Transaction Class value to a submitted job. That allows mapping to Service Class and Reporting Class:

Dispatcher Function

xJCL

Control Region

Servant Region(s)

Batch Application

WLM

WLM

Classify Work

DispatchSubmit

If (rule) then TC=tclass● Job class from xJCL● Jobname● Application Name● Submitter ID● other criteria

TCLASS

Assigned Service Class and/or Reporting Class

Two use-cases for this:

Assign separate WLM Service Classes

Have jobs dispatched into separate servant regions where WLM can manage to the goal

Assign separate WLM Report Classes

This allows categories of Java Batch jobs to have system resource information accumulated under the report class for RMF reporting

WebSphere Application Server for z/OS has extensive integration with z/OS Workload Manager (WLM) for the

purposes of request classification and priority management. The WebSphere Java Batch function can take

advantage of this and classify batch jobs during submission and dispatch.

Note: this topic requires some familiarity with z/OS WLM to understand the details of what's being said here. If the

audience is non-z/OS people that familiarity may not exist. Explain WLM as function of z/OS that controls access

to system resources -- CPU, memory and I/O -- to manage all the work on the system to the relative priority goals

defined. The "Service Class" is what carries the goal definition. When two pieces of work are competing for the

same system resources, then WLM gets involved and works to make sure the defined goals are met. Higher

priority goals take precedence over lower priority goals, but WLM also makes sure lower priority is not starved

altogether. WLM is integrated deeply into the z/OS operating system, which is how it can control access to system

resources to manage competing work to the defined goals.

The starting point for this is the WebSphere Java Batch "Dispatcher Function" using rules you supply to assign a

"Transaction Class" name to a submitted job. Various criteria is available. The easiest to think of is "If Jobname =

XYZ, then assign Transaction Class = ABC." Once the Transaction Class (TC) name is assigned, that is passed

to the endpoint server where the batch application resides. The Controller Region works with WLM to classify the

work and the TC name is passed into WLM. That results in a Service Class and/or Report Class being assigned to the job.

Once the Service Class or Report Class has been assigned, then two use cases suggest themselves. They are

spelled out on the chart above. Of the two, the use of separate Report Class is the more common. This would allow batch jobs to be grouped into a relatively few categories of Report Class, which WLM then uses to

accumulate system resource statistics for all work operating under those Report Classes. So imagine 1000

different batch jobs, with 700 categorized under Report Class A and 300 under Report Class B. Those batch jobs

run at various times throughout the day. RMF may then be used to report on the system resource statistics for

everything that ran under A and everything that ran under B. That is a very good way to capture details on a group

of jobs, rather than having to accumulate information individually.


49


Gaithersburg, MD49


Job Usage Information -- SMF 120.9 on z/OS

Positioning …

In addition to capturing job statistics in a relational table (all platforms), on z/OS the SMF facility can be used to capture batch job information.

Job Number

Submitter ID

Total CPU used

Job state

Node/Server

Job start time

Last update time

Accounting string

Job Number

Submitter ID

Total CPU used

GP Time used

Job state

Node/Server

Job start time

Last update time

Accounting string

All Platforms z/OS SMF 120.9

The information provided is very similar between "all platforms" and "z/OS SMF 120.9"

SMF 120.9 augments CPU information with GP usage. You can derive offload: Total - GP = Offload

If z/OS SMF is part of other job accounting and chargeback purposes, then SMF 120.9 for WebSphere Java Batch is simply part of that.

An example of taking advantage of the platform services

Being able to account for what batch jobs were run, who ran them, when they ran, and how much resource they

consumed is an important part of managing any batch environment. The WebSphere Java Batch function

provides accounting information on all supported platforms by writing the accounting information into a defined

relational database table. The information captured is shown in the left side of the chart above.

Note: that includes z/OS. If you have WebSphere Java batch on different operating systems and you wish to

consolidate to a common set of accounting information, you may have the z/OS capture to a relational table in

exactly the same manner that it does on other platforms. You may use SMF 120.9 in place of or in addition to this

common accounting information capture.

SMF 120.9 in WebSphere Java Batch on z/OS provides two things:

1. A way to use SMF 120.9 for Java Batch if SMF is used for other job accounting in the mainframe environment.

In other words, by providing SMF 120.9 for Java Batch it allows accounting for this to easily merge into existing

skills and existing processes.

2. The ability to derive the offload achieved in Java Batch processing. The SMF 120.9 accounting information

extends the CPU used value to include GP used. Therefore, the offload amount is a derived value -- subtract

GP from Total to arrive at the offload amount.


50


Gaithersburg, MD50


Positioning WebSphere Java Batch

Summary …


JVM Model

Batch Runtime WebSphere Java Batch

COBOL Invocation Both WaysJava invoke COBOL or COBOL

invoke Java; z/OS 2.1 adds PL/I as wel. You write the JNI glue code

One WayJava invoke COBOL but not

the other way around. JNI glue code supplied with runtime.

Bit-mode 31-bitExpectation is it will be

used with COBOL

31-bit or 64-bit31-bit if COBOL container;

64-bit possible if not

JVM LauncherImplies JVM instantiation and tear-

down for each invocation

Long Running JVMBuilt on WAS server model where

servers stay up for as long as needed

Data Access Full Java EEAny data access protocol supported by Java EE WAS itself (JDBC, JCA,

JMS, WOLA, etc.)

JDBC and tVSAMIn z/OS 2.1 support for JDBC T2

and transactional VSAM.

Batch Services Full Java BatchEverything Batch Runtime has

plus Dispatcher and PJM

Subset of Java BatchIn z/OS 1.13 TX services; in z/OS 2.1 very close to full Java Batch, but no PJM

Programming Model WebSphere Java BatchFull Java Batch programming

model. Ability to wrapper Java main(). Full Java EE runtime.

WebSphere Java BatchIn z/OS 2.1 the support is based on Java

Batch. Difference: no PJM. Also, only JDBC T2 and tVSAM, no other Java EE access.

Here is the positioning chart for WebSphere Java Batch compared to the Batch Runtime. We are not showing

BPXBATCH and JZOS here because those are JVM launchers without a batch support infrastructure provided,

but the Batch Runtime is based on WebSphere Java Batch so a comparison is useful.

Note: the JZOS toolkit, which provides access to z/OS services, may be used by either the Batch runtime or

WebSphere Java Batch.

The general positioning is this -- if a JVM launcher technology fits the nature of your batch requirements, and you

do not need access to WAS classes or a Java EE runtime, then the Batch Runtime can work for you. But if you

wish a more comprehensive batch runtime built on a long-lived Java EE platform, then WebSphere Java Batch is

the choice.


51


Gaithersburg, MD51


Section Summary

CICS, DB2 …

A very broad summary of the WebSphere Java Batch solution:

WebSphere Java Batch represents a full-function Java Batch framework and runtime model

Much more batch function than BPXBATCH or JZOS

It is built on WebSphere Application ServerWhich means all the functional value statements of WAS accrue to the Java Batch environment. On z/OS, all the "Why WAS z/OS" value statements accrue as well.

Integration with Enterprise SchedulersIt provides a messaging-based interface to the Dispatcher function so Enterprise

Scheduler functions can treat WebSphere Java Batch as any other batch job. This allows WebSphere Java Batch to be part of larger orchestrated batch flows.

Here is the summary for this section.


52


Gaithersburg, MD52


Java Batch in CICS, DB2Exploring how Java Batch is implemented in those key data systems


53


Gaithersburg, MD53


WebSphere Java Batch and Access to CICS

Feature Pack for Modern Batch …

Imagine a WebSphere Java Batch application needing access to CICS for data required to complete the batch job. How can that CICS access be accomplished?


EJB Container

Web Container

Batch Container


CICS Program

CICS Region

Dispatcher FunctionxJCLxJCL


What options exist for this interaction?

Web ServicesUbiquitous, open-standard protocol that spans platforms

Overhead per invocation somewhat high, particularly if transaction required. Not a good solution if latency per invocation a concern.

CTG JCAProven WAS-to-CICS interaction pattern.

WAS-to-CICS global TX

If EXCI, then 32K limit

If IPIC, then TCP stack code path is invoked

WOLAHighly efficient WAS-to-CICS interaction pattern

WAS-to-CICS global TX

Same LPAR required

The objective of this chart is to imagine how a Java batch program running in WebSphere Java Batch might gain

access to a CICS program as part of the batch processing. The chart is showing three potential means of

accessing CICS. All are capable of performing the task, though each has its own advantages and disadvantages

associated with it.

Note: the purpose of this is to set the stage for the next function we'll discuss, which is a means of extending the

WebSphere Java Batch model to include batch endpoints that run in the CICS region. That puts the Java batch

program right next to the CICS resources and the ability to directly invoke the JCICS interface to access CICS

services. When a Java batch architecture calls for access to CICS resources, the next feature may well be the

right solution.


54


Gaithersburg, MD54


CICS TS Feature Pack for Modern Batch V1.0

Programming Model …

This allows a CICS region to become an job endpoint for a WebSphere Java Batch dispatching server. Puts batch logic much closer to the CICS data:



FP Modern Batch 1.0

Java Batch Application CICS

Program

JCICS Interface and Services

CICS Region TS4.2 or TS5.1

XML file in CICS Feature Pack setup lets that batch container communicate with the WAS Dispatcher Function host and port. That lets the Dispatcher know the

endpoint is present and what batch applications are deployed.

The application in the CICS batch container has a programming model very similar to that used by WebSphere Java Batch. Benefit is access to JCICS services and

very close proximity to the CICS program the batch application requires.

The batch job management is maintained in WebSphere Java Batch. The dispatching may take place to either WAS endpoints or CICS endpoints or both.

WAS V8.5 or aboveAny Platform

Communicates availability and applications deployed in container

xJCL submitted with application in CICS container gets dispatched

xJCL submitted with application in WAS container gets dispatched

The CICS TS Feature Pack for Modern Batch installs into the CICS region and presents itself to a WebSphere

Java Batch dispatcher as another endpoint able to dispatch work to. That provides Java batch management in the

hands of the WebSphere Java Batch runtime model with CICS being an endpoint.

The Feature Pack for Modern Batch function is configured to communicate over the network to the Dispatcher,

telling the Dispatcher about the CICS Feature Pack presence and the Java batch application deployed there.

Then when an xJCL is submitted to the Dispatcher for the Java batch program deployed in the CICS Feature

Pack, the Dispatcher communicates across the network to invoke and monitor the progress of the batch program.

Note: think of the CICS Feature Pack for Modern Batch as just like a WebSphere Java Batch endpoint function,

except it runs in CICS. It uses the same programming model as WebSphere Java Batch. The Java batch

program may then access CICS services directly. This is ideal for cases where the Java batch programs calls for

intensive interaction with CICS.

The integration with Enterprise Scheduler discussion earlier is exactly the same in this scenario since the

WebSphere Java Batch dispatcher is the job management function. WSGRID interacts with the Dispatcher, and

the Dispatcher interacts with CICS to run the batch program. From the Enterprise Scheduler point of view it's all

hidden behind the WSGRID proxy job.


55


Gaithersburg, MD55


Programming Model for CICS Modern Batch FP

Positioning …

The programming model is based on the WebSphere Java Batch model. There are a few restrictions plus some added functions:

Essential Programming ModelEverything discussed in the earlier section on WebSphere Java Batch applies to the CICS Modern Batch FP as well, except Java EE functions and those things noted below.

Some Batch Data Stream Patterns Not SupportedThe following patterns of WebSphere Java Batch are not supported in the CICS Modern Batch Feature Pack environment:

JDBCReader and JDBCWriterLocalJDBCReader and LocalJDBCWriter

See below ... Feature Pack provides its own JDBC patterns

Additional Batch Data Stream Patterns ProvidedThe CICS Modern Batch Feature Pack adds the following BDS patterns:

VsamKsdsReaderImpl -- read from a VSAM KSDS as input data to the job.

VsamKsdsUpdaterImpl -- update in a VSAM KSDS file.

CICSJDBCReader -- retrieve data from a database using a Java Database Connectivity (JDBC) connection.

CICSJDBCWriter -- write data to a database using a Java Database Connectivity (JDBC) connection.

In addition, the CICS Modern Batch Feature Pack provides access to the full set of JCICS APIs

Here we make the case that the programming model for the CICS Feature Pack for Modern Batch is compatible

with that of WebSphere Java Batch. It's not entirely compatible, but it's very close.

Note: The JVM Server inside of CICS does not provide a Java EE runtime model. It's Java SE. So in this sense

the programming model is like that provided in the Batch Runtime. So while not exactly the same programming

model, the essential structure of represented earlier -- job steps acting upon a data record, looping controlled by

the batch container, checkpoint and commit/rollback handled by the container -- is the same.

There are restrictions on some of the Batch Data Stream patterns supported. The WebSphere Java Batch JDBC

patterns are not supported, but the CICS Feature Pack supplies its own patterns for JDBC as well as VSAM. And

the JCICS APIs are also available to the Java batch program that runs in the Feature Pack.


56


Gaithersburg, MD56


Positioning CICS Modern Batch Feature Pack

DB2 Stored Procedures …


WebSphere Java Batch Modern Batch FP

JVM Model Long Running JVMBuilt on WAS server model where

servers stay up for as long as needed

Long Running JVMBuilt on CICS JVM server model; JVM

stays up for as long as needed

Programming Model WebSphere Modern BatchCI or Transactional Batch

WebSphere Modern BatchCI or Transactional Batch

Multi-Step YesMulti-step jobs supported

YesMulti-step jobs supported

Job Dispatching Job Dispatching FunctionJob dispatching is handled

through the WebSphere Java Batch "scheduler" interface

Job Dispatching FunctionJob dispatching is handled

through the WebSphere Java Batch "scheduler" interface

Parallel Job Manager YesSupports WebSphere Java Batch

PJM interfaces and operation

YesSupports WebSphere Java Batch

PJM interfaces and operation

Enterprise Schedulers YesIntegration through WSGRID and

MDB interfaces to dispatcher function of WebSphere Java Batch

YesIntegration through WSGRID and

MDB interfaces to dispatcher function of WebSphere Java Batch

This is the positioning chart for WebSphere Java Batch compared to the CICS Feature Pack for Modern Batch.

You'll notice that the two are very similar. Nearly identical, in fact.

Note: this chart does not reflect the differences highlighted on the previous chart; that is, Batch Data Stream

differences and the fact the CICS JVM Server is Java SE not Java EE. Otherwise, very close functionality

between the two.

If you pause and think about it, this makes sense -- the CICS Feature Pack for Modern Batch is designed to be an

extension of the WebSphere Java Batch runtime model. If the two endpoint designs were radically different the

"extension of WebSphere Java Batch" model would break down. But because it's designed to be highly

compatible, the model holds.

It's important to understand the CICS Feature Pack for Modern Batch is operational only if there's a WebSphere

Java Batch dispatcher function deployed somewhere that serves to dispatch and monitor the Java batch running

in CICS. That WebSphere Java Batch dispatcher may be anywhere on the network, on z/OS or non-z/OS. But it

has to be somewhere as without it there would be no way to manage and run the batch programs in the Feature

Pack.

So again, think of the CICS Feature Pack for Modern Batch as a way to extend the management domain of a

WebSphere Java Batch dispatcher. Then the design becomes clear -- Java batch running on server endpoints

where it makes the most sense, with a common control and monitoring manager (the Dispatcher) at the heart of it.


57


Gaithersburg, MD57


High Level: DB2 Stored Procedures Written in Java

Summary …

Stored procedures allow for compact and efficient SQL calls from Java. Writing them in Java allows the execution of the stored procedure to be offloadable:

DB2 for z/OSV7 or above


EJB Container

Web Container

Batch Container

Java Batch Applications

z/OS Operating System

System z HW Platform

Java source for DB2 Stored Procedure

Compiled Java classDB2 Stored Procedure

WLMWLM

StoredProcedure

Address spaces established by WLM to run JVM and Stored Procedures

NUMTCB in proc determines how many JVMs per address space; JVMs single threaded and re-used

CREATE PROCEDURESQLAPPL ENVJCL Proc

Reference: http://www.redbooks.ibm.com/abstracts/sg247604.html

Details behind all this, obviously ... key point is DB2 supports Java as a language for Stored Procedures (User Defined Functions as

well) and those may be called by batch processes

This shows WebSphere Java Batch, but any batch environment capable of calling SP in DB2

may exercise Java-based Stored Procedures

It may seem odd to include DB2 Stored Procedures as part of this discussion because Stored Procedures are not

really batch programs in the sense we're using that term in this presentation. But Stored Procedures may well be

part of the DB2 process used by the Java Batch program calling the Stored Procedure.

Note: this topic is presented to complete the picture and show how even DB2 Stored Procedures can run using

Java with offload to the zAAP processors. The WebSphere Java Batch model (or any Java batch model we've

discussed that can call DB2) doesn't really know or care that the Stored Procedure is written and runs in Java. But

you care because of the benefits it provides -- leveraging Java skills, Java tooling, and specialty engines.

When a Java Stored Procedure is called, DB2 works with WLM to create address spaces inside of which JVMs

are established to run the code. WLM controls how many address spaces based on goals defined, and within any

given address space the parameter NUMTCB is used to determine how many JVMs to host within the address

space. Each JVM is single-threaded and is re-used after the Stored Procedure has been run.

The Redbook referenced in the URL on the chart is an excellent source of information about this topic and, in fact,

the much broader topic of Modern Batch.


58


Gaithersburg, MD58


Section Summary

Decision Management …

A very broad summary of the CICS and DB2 section:

The CICS Modern Batch Feature Pack extends WebSphere Java Batch management/execution realm

It allows CICS to participate as a WebSphere Java Batch endpoint server.

A CICS endpoint puts batch logic closer to CICSWhat work should run in the CICS endpoint is a function of the batch job and the

nature / intensity of its CICS interactions

DB2 Stored Procedures and JavaNot really "batch" in our normal sense, but it can part of a larger batch process.

Stored procedures can make batch execution more efficient, and Java stored procedures allow for further exploitation of Java on z/OS

The summary chart.

What this section set out to do was to show how "Modern Batch" can extend beyond the WebSphere Java Batch

model we showed earlier. That highlights the fact that when designing a Modern Batch environment, there are

many options available to you to use what's appropriate given your needs.

Note: what about IMS? It too has facilities for Java batch. It's not included in this presentation because the topic

is fairly extensive and experience has shown those interested in IMS are typically interested in a more detailed

treatment of the topic. So if you are interested in IMS, by all means please contact your IBM IMS expert to discuss

the matter.


59


Gaithersburg, MD59


Batch and Decision ManagementIntegrating Java Batch with WODM


60


Gaithersburg, MD60


Batch Step Decision Logic -- In Step or Outside Step?

Operational Decision Manager …

Imagine a batch job that reads data records and acts upon those records based on a set of decisions (or rules). Where would you prefer those rules be?

Job

Step

Read

Process

Write

Rules

Step

End

Decision logic written within the batch step

Job

Step

Read

Process

Write

Step

End

Rules Execution

Server

Call

Result

Rules Designer Tooling

Rules Repository

Rules

All else equal, it provides greater flexibility of design,

change and execution to externalize the rules

What happens when a batch program hits a point within its processing where a decision about how to process

needs to be made. Where is that decision logic best housed?

One place is in the batch program itself. And in fact in the past that's exactly how it's been done. Many thousands

of COBOL batch programs in existence today have business decision rules hard-coded deep into the batch

processing logic. Changing those rules when needed is complicated and time-consuming.

Which is why rules engines have become quite popular -- they provide a means of externalizing rules design and

rules execution from the mainline program. That enhances flexibility and change agility.

Note: what the chart above is showing is how a rules engine such as IBM's Operational Decision Manager can be

used with batch programs to take advantage of externalized rules. The intent is to foster a discussion of decision

logic within batch processing today and explore how a product like IBM's Operational Decision Manager may

provide additional value.


61


Gaithersburg, MD61


IBM Operational Decision Manager

RES Options …

Provides real-time handling of business rule execution and situational awareness via correlation of business events received:

Decision Model:Given a snapshot view of data, determines best course of action at a specific point in a process or application

Main Purpose:To automate a decision based on a combination of factors such as business policies, regulations, and best practices.

Example:If the customer is not primary card holder and age is less than 21 then freeze account and notify primary card holder

Time-Based Pattern Detection:Correlating events as data is in motion

Main Purpose:To determine what of interest is transpiring and coordinate one or more responses by other systems or generate alerts to people

Example:If more than 2 customer withdraws in an ATM are done in the same day and the 2 ATMs are from 2 foreign countries then Investigate possible fraud. Reduce cash redraw max amount to 100$

Business Rules Business Events

IBM's Operational Decision Manager (ODM) provides two key functions -- handling business rules (the point of the

previous chart) and business events.

Note: this chart is included to give ODM proper credit for the scope of what it does. The main focus for Java

batch is more the business rules capability of ODM. But business events management may well be of interest for

other aspects of the business. So this chart simply paints the broader picture.


62


Gaithersburg, MD62


High Level: Rules Execution Server Options

JSR 352 …

The Rules Execution Server (RES) has a few means of placement in z/OS architecture:

zRES

zRES

RES on WAS for z

COBOL

Java

COBOL

IMS

COBOL

Batch

CICS Region

WebSphere Application Server for z/OS

Started Task

The Rules Execution Server may run either in a CICS region, as a

standalone started task, or inside the Java EE runtime of WAS z/OS

Our focus is on how a Java batch application in WebSphere Java Batch may access the RES, and it's by running it in WAS as well.

Invocation patterns on WAS include EJB and remote EJB.

Depending on proximity of Java Batch to RES, one of those two

would be optimalOther interfaces: Web Services and JMS/MDB

WO

LA

This is a somewhat busy chart. It is intended to show the ways a batch program -- either COBOL or Java -- may

access the Rules Execution Server (RES) of Operational Decision Manager.

Two flavors of zRES are shown here -- one is native code (not Java), and the other is Java and runs in

WebSphere Application Server. Since our primary focus is on Java batch, the invocation path from that into RES

for WAS on z would be direct invocation of the RES EJB from the Java batch program. Other interfaces exist,

including Web Services and JMS/MDB, so there's flexibility in the design.

If the design called for RES to be run in WebSphere Application Server for z/OS as a common hosting platform for

ODM on the mainframe, then COBOL access into the WAS z/OS runtime is provided by WebSphere Optimized

Local Adatpers, or WOLA. WOLA is a high-speed cross-memory copy mechanism that is very efficient for calls

such as decision rule calls.

For more on WOLA -- ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP101490

For more on ODM on z/OS -- ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP102291


63


Gaithersburg, MD63


Open Standard Java BatchA look at the JSR 352 specification


64


Gaithersburg, MD64


JSR 352 - "Batch Applications for the Java Platform"

Generic Diagram …

The V1.0 final release is dated April 18th of 2013. IBM served as specification lead on this JSR. It is available for download from the web at the following URL:

http://jcp.org/aboutJava/communityprocess/final/jsr352/index.html

The document has a very nice section titled "Domain Language of Batch" which provides an overview of key concepts and key

terminologyAs well as detailed sections on the specification interfaces and other details of the specification

If interested in this JSR, download and review is encouraged

An open standard for Java batch has emerged. It is referred to as JSR-352, and that specification defines a set of

functions and interfaces a standard Java batch platform will implement. Representatives from several companies

participated in the JSR development, including IBM. IBM's Chris Vignola served as specification lead for this

project.

The JSR final release document may be downloaded at the URL presented on the chart.

Note: the JSR document is surprisingly approachable and readable, particularly the "Domain Language of Batch"

section, which provides an excellent overview of key concepts. If you're at all interested in the JSR 352 topic,

downloading this document and reading it is highly recommended.


65


Gaithersburg, MD65


Generic Diagram for Domain Language of JSR 352

Comparison …

The diagram is generic but useful in that it provides a few key conceptual building blocks on which the rest of the specification relies:

JobOperator Job Step

ItemReader

ItemProcessor

ItemWriter

JobRepository

1 * 1

1

1

1

A job has one to many steps

Retrieval of input, one item at a time

Business processing for an item

Output of the step -- item or chunk

Interface to manage aspects of job processing

Holds information about jobs currently running and jobs that have run in the past

Concepts here are applicable to batch of any flavor -- historic COBOL, WebSphere Java Batch, JSR 352.

The chart above comes from the JSR-353 final release document. It provides an abstract representation of a

batch structure with the terminology used in the JSR.

This maps well to WebSphere Java Batch because the concepts employed by WebSphere Java Batch are nothing

new. Batch processing has been around for a very long time and the concepts have fairly well settled around a

recognizable pattern. The terminology used is not always the same, but the fundamental concepts are.

Note: think of the JobOperator as relating to the Dispatcher function of WebSphere Java Batch. The Job and

Step blocks in the picture map to the Job and Step representation in xJCL we spoke of earlier. The ItemReader,

ItemProcessor and ItemWriter concepts above map to the representation of the batch step where we spoke of a

data record being retrieved and acted upon. The JobRepository concept maps to the relational tables used by

WebSphere Java Batch.

It's important to recognize that the JSR-352 standard is relatively new, and WebSphere Java Batch does not at

this time implement the standard as released. The preceding paragraph was meant simply to show how the

concepts of JSR-352 map to the structural concepts of WebSphere Java Batch.

So that naturally leads to the question ... how would WebSphere Java Batch compare to the specification as

drawn up in JSR-352? The next chart provides that.


66


Gaithersburg, MD66


JSR 352 Comparison to WebSphere Java Batch

Possible Integration …

A good deal of functional similarity with a few points where each has differential:

From Chris Vignola's IMPACT 2013 Presentation, TAW-1443

This chart shows a set of features and how the JSR 352 standard addresses those features and how WebSphere

Java Batch does. There's a good many features where "yes" applies to both columns.

The three functions supported by JSR 352 but not WebSphere Java Batch are concurrent steps (WebSphere Java

Batch step execution within a job is sequential), restart substitution (the ability to substitute in new values on job

restart; WebSphere Java Batch has a restart function, but does not allow value substitution on restart), and CDI

refers to dependency injection, which WebSphere Java Batch does not support.

On the other hand, WebSphere Java Batch supports several things the JSR 352 spec does not call out -- a Job

Dispatcher function (JSR 352 defines a JobOperator interface but how that's implemented is not declared), job

logs (JSR 352 does not have specific details on job log output formatting), and Distributed Parallelization, which is

the PJM function we discussed earlier.

Note: the objective here is not to claim one is better or worse than the other. IBM understands the value of open

standards and embraces their development and adoption (IBM lead the JSR-352 specification). The objective

here is first and foremost to stress the commonality of JSR-352 and WebSphere Java Batch at the functional

level, and then to provide an understanding of where differences exist.


67


Gaithersburg, MD67


Theoretical JSR 352/WebSphere Batch Integration

References …

A topology picture showing both models working cooperatively:

From Chris Vignola's IMPACT 2013 Presentation, TAW-1443

Common Batch Container

WebSphere Java Batch Applications



JSR 352 Java Batch Applications

JSLJSL

JSR 352 JSL

Common Job Repository

Common Job Logging

This implies no promise of future delivery ... that said, the model shown above is well within the possible given the nature of JSR 352 and the

similarities with the WebSphere Java Batch model.

WebSphere Java Batch as well as JSR 352

JobOperator function

The chart above provides a potential means by which existing WebSphere Java Batch can be enhanced to

provide both JSR-352 (new function) and WebSphere Java Batch (protect existing investment).

Note: normal disclaimers apply ... this does not represent a promise to provide function ... IBM reserves the right

to modify its plans at any time. Please, this chart is intended to suggest a potential path, but is not a promise of

one.

The previous chart illustrated how the two approaches are more similar than different. That means there's good

opportunity to have concurrency of supported function within the framework offered by WebSphere Java Batch.

Take for example the current Job Dispatcher function. It currently accepts only xJCL. But in theory it could be

enhanced to accept both xJCL and the JSR 352 JSL (Job Submission Language ... also XML, but with a syntax

different from xJCL). Similarly, the container function currently implemented to the WebSphere Java Batch model

could be enhanced to support both runtime models.

The key point here is that IBM was active in the formulation of JSR 352 and thus is makes sense that IBM would

be active in its adoption. How exactly that adoption is implemented and the timeline for implementation is not

spelled out here.


68


Gaithersburg, MD68


ReferencesA guide to reference information for further details


69


Gaithersburg, MD69


References

http://www.ibm.com/developerworks/systems/library/es-java-batchz.html

JZOS vs. BPXBATCH

http://www.ibmsystemsmag.com/mainframe/trends/whatsnew/java_compiler/

SDK 7 features with EC12

http://www.ibm.com/systems/z/os/zos/tools/java/products/jzos/overview.html

JZOS

z/OS 2.1 announcement with information on Batch Runtime improvements

http://www.ibm.com/common/ssi/rep_ca/3/897/ENUS213-013/ENUS213-013.PDF


http://www.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP101783

CICS Transaction Server Feature Pack for Modern Batch V1.0

http://www-01.ibm.com/software/htp/cics/batch/

A few references. Other URLs were provided throughout the charts.

End of DocumentEnd of Document

ibm java batch solutions - ibm www page · java on z/os jvm launcher solutions websphere java batch...

Documents