Batch Java Overview

Gregg Lippa

Senior Technical Analyst

Themis Education

Themis, Inc.

glippa@themisinc.com

Visit us at: www.themisinc.com

Also: www.themisinc.com/webinars

Twitter: @ThemisTraining

This Webinar Addresses These Questions• What are Java batch applications and why should you use them?

• What scenarios lend themselves to Java batch?

• How do you create and deploy Java batch applications?

• What role do Eclipse and Ant play?

• How are JCL and Unix shell scripts used to execute them?

• What are BPXBATCH, BPXBATSL and JZOS?

• How do you access sequential, PDSE and VSAM files?

• How is logging and console communication handled?

• Can I create Java Record Classes from COBOL copybooks?

• How does remote debugging come into play?

• Can zIIP reduce execution cost of batch Java applications?

• We will have very little time for questions

– Please submit questions via email

to glippa@themisinc.com2

Legacy Systems

vs

Stand-alone Java

Java Stand-Alone Applications

• A Java stand-alone application

– Has a main method

• Similar to a main entry point of a traditional COBOL

– Runs in its own JVM

• Does not require a J2EE server environment such as WAS *

– May use the Java Native Interface (JNI)

• Can be used to combine Java with COBOL and to

access z/OS UNIX Assembler Callable Services

– Is autonomous and needs a trigger to be launched

• Via Unix command or JCL

4* WAS – WebSphere Application Server by IBM

Batch Applications

• Features of batch applications

– Start and stop time are not critical

• Not expecting response times measured in seconds

– No terminal or browser is waiting for output from the job

– Typical process a large workload

• Repeat a task thousands or even millions of times

– May help achieve more efficient processing

• Throughput can be increased via batch execution

– Maya use advanced workload management techniques

5

Legacy System Lifecycle• Application lifecycle continues beyond development and implementation

– Includes maintenance, modernization and replacement

– The modernization and replacement phases are particularly

relevant for stand-alone Java applications on z/OS

• Eventually, ongoing maintenance is no longer acceptable

– Obsolete user interfaces, incompatible operating environments, etc.

• One simple approach is to implement a wrapping layer

– Provides newer interfaces using techniques such as screen scraping

• Advantages of screen scraping

– Tools exists to facilitate transformation of existing applications

– Protects the investments in existing applications

– Is less risky than replacement with a newer system

• However, underlying applications are not modernized

– Makes these solutions difficult to maintain

6

Legacy System Replacement

• Replacement - last phase of legacy system modernization process

– Legacy system is completely replaced with a new system

• Involves greater risk

– No guarantee that new system will provide better service to users

in spite of good intentions and design of the new application

• The legacy systems must not be dismissed lightly

– They have been supporting the business successfully for many years

• New systems may need to access resources based in z/OS

– Such systems may also benefit from Java batch components

– Opportunities to modernize application portfolio exist here as well

7

Considerations for ReplacementFactors to consider when replacing a legacy application:

1. Availability of skills, especially looking toward the future

– Fewer COBOL developers available as time passes

– More Java developers coming online every day

– More tools to increase developer productivity in the Java realm

2. Integration with other systems over the network

– Java and eclipse-based IDEs offer proven integration frameworks

• Increases cost-effectiveness of developing new applications

– Java in particular supports a wide range of APIs

• E.g. JDBC, JMS (messaging), XML parsing, Web Services support

3. Performance as measured by application throughput

– Processing a large volume of data within a certain time frame

– Java has been improving in this regard (e.g.: JIT compiler)

4. Portability allows an application to run in other operating environments

– Applications may move off z/OS

– Java: “write once, run anywhere”

8

Why Java for Stand-Alone Applications?1. Functionality such as sending e-mail or updating a remote database in a batch job

– Availability of existing reusable objects (utility classes, class libraries, etc)

2. XML processing and Web Service support

– Availability of APIs like SAX and DOM to work with XML documents

3. Reuse J2EE online program logic to manage complex and heavy workloads

– Longstanding record of reliability and manageability

4. Migrating Java applications for QoS and server consolidation

– IBM Java SDK for z/OS is certified for Java compatibility.

5. Availability of skill set for the future

– Easier to find Java developers than to find COBOL developers

– A wide range of libraries and APIs are available for Java development

– Powerful IDEs improve productivity and provide GUI-based debugging

6. Java applications can run most instructions on zIIP processor

– COBOL programs can only use a General Purpose processor

– Can result in significant cost savings 9

Java Stand-Alone Application Scenarios

• Some scenarios that lend themselves to considering Java:

– Data processing jobs

• Large volumes of data processed in batch

– Flat files, VSAM data sets or DB2 data

– Messaging to communicate with other environments

• Can use JMS to feed a remote queue

– Allows an application to integrate with other environments

– Generating and printing reports

• Large amounts of print work to be completed in batch

– May utilize print servers, terminal servers and HTTP servers

– Independent Software Vendor (ISV) applications

• Software solution providers may deliver Java batch applications

to support the industry-specific needs of your organization

10

Other Java Batch Capabilities

What you can do with Java on z/OS in a batch environment?

– Call stored procedures using JDBC or SQLJ

– Read from and write to various types of files

• Including UNIX files, MVS data sets, PDSEs and VSAM data sets

– Call programs in CICS or IMS using Java Connector Architecture (JCA)

– Use protocols such as TCP/IP and HTTP to access the network

– Perform CPU-intensive calculations

– Integrate with other programming languages using

• Java Native Interface (JNI)

• Language Environment (LE)

11

Building and Deploying

Java Batch Applications

Automate Deployment and Debugging• Application must be deployed, and eventually debugged, on z/OS

– Tools help automate application deployment and remote debugging

• Use Eclipse to build and deploy stand-alone Java applications

– Eclipse Ant support helps automate application packaging and deployment

• Compiled classes are packaged into jar files

– Deployed to USS on z/OS using FTP tools

• Many other files may be needed for an application

– e.g. JCL, USS Shell script, properties file, configuration files, XML files

• Moving files to z/OS may require managing code page conversion during ftp

– Tedious and error-prone

– Automate process to ease application development

• Debug applications that are running on z/OS

– Use Eclipse Java debugger to debug Java programs running on z/OS

13

Eclipse-Based Ant Support

Use Apache Ant to automate Eclipse-based Java Z/OS development:

1. Local compile of Java app, including use of Z/OS-specific APIs

2. All resulting *.class files are packaged into a .jar file

3. The new .jar file is uploaded via FTP into Unix System Services (USS)

4. Locally developed JCL is uploaded to support batch processing with JZOS

5. Uploaded batch job is submitted via FTP

– Invokes uploaded Java app in JVM via JZOS launcher

– Runs in the same address space as the Java application

6. JES output of Java batch job is retrieved via FTP to Eclipse

7. App can be debugged in Eclipse

– Uses remote debugging through z/OS JVM

14

Apache Ant and Eclipse Support

• Apache Ant is an open source, Java-based build tool

– Ant buildfiles are XML files containing scripts (targets with tasks)

– They can be created and run from Eclipse workbench

– They can operate on resources in the workspace and in the file system

– Output from an Ant buildfile is displayed in the console view

• Ant editor provides specialized features for editing Ant buildfiles

– Can easily build scripts to develop Java applications for z/OS

• Easy to run an Ant buildfile in the workbench with a few clicks

– Can run any targets in Buildfile in any reasonable sequence

15

Build Properties File• Separate properties file is used to support Ant buildfile

– Externalize to promote reusability of ant build

– Tailor properties file for your own site and purposes

– Makes Ant buildfile more generic and flexible

– Provides centralized place to customize Ant script

– Can update properties file and use Ant buildfile without any changes

16

zOS.hostname = p390.themisinc.com

zOS.ftp.userid = USER123

zOS.ftp.password = secret

project.app.jar = MyApp.jar

project.Jar.dir = jarFolder

project.JCL.dir = jclFolder

project.shscript.dir = scripts

project.misc.dir = misc

zOS.jar.dir = /u/id123/jbatch/java/sample

zOS.JCL.dsn = ${zOS.ftp.userid}.SOMEAPP.CNTL

zOS.script.dir = scriptFolder

zOS.misc.dir = ${zOS.ftp.userid}.SOMEAPP.DATA

project.misc.dir = supportingFiles

Build and Deploy Ant Script

<project name="deploy" default="all">

<property file="${user.home}/build.properties"/>

<property file="../build.properties" />

<property file="./build.properties" />

<target name="all" depends="compile, deployJar, deployJcl, submit" />

<!-- for compiling if not done automatically by IDE (Eclipse) -->

<target name="compile">

<mkdir dir="bin"/>

<javac destdir="bin" classpath="lib/jzos.jar" debug="on">

<src path="src"/>

</javac>

</target>

17

• This Ant build file invokes all necessary steps of development cycle:

1. compile target – performs local compile on workstation

2. buildJar target – builds a jar file containing all Java classes

3. deployJar target – uploads the jar file via FTP

4. deployJcl target – uploads JCL via FTP

5. submit target – submits the Job via FTP

• Java Ant task invokes a Java app in submit.jar that

submits jobs via FTP and retrieves the JES output Simplified

and streamlined sample

Ant buildfile

Build and Deploy Ant Script (page 2)<target name="buildJar">

<mkdir dir="deploy"/>

<jar destfile="deploy/${jarname}">

<fileset dir="bin"> <include name="**/*"/> </fileset>

</jar>

</target>

<target name="deployJar" depends="buildJar" >

<echo message="Copying files to ${server}:${appl.home}..."/>

<ftp server="${server}" userid="${userid}" password="${password}"

remotedir="${appl.home}" depends="no" binary="yes" umask="002" verbose="yes" >

<fileset dir="deploy" includes="" casesensitive="yes" />

</ftp>

</target>

<target name="deployJcl" >

<echo message="Copying JCL members to ${server}:${jcl.dsn}..."/>

<ftp server="${server}" remotedir="${jcl.dsn}" userid="${userid}"

password="${password}" binary="no" verbose="yes" depends="no" >

<fileset dir="jcl" > <include name="*" /> </fileset>

</ftp>

</target>

<target name="submit">

<java jar="submit.jar" fork="true">

<arg line="${server} ${userid} ${password} ${jclPath} ${debug} ${waittime}" />

</java>

</target>

</project>

18

Requires a

different set of properties

(not shown)

Submitting Jobs and Controlling Output

• Customize the project and deploy it

– Right-click deploy.xml and select Run As → Ant Build...

– Select the all target

– Click Run

• The Eclipse console displays the status of each Ant build step

1. All necessary files are compiled and uploaded to the host

2. The job is submitted via FTP

3. JES SYSOUT is displayed on the console

4. Job is purged

19

Java Batch Execution

BPXBATCH, BPXBATSL and JZOS

Java on z/OS – UNIX Environment• The Java Virtual Machine runs in the z/OS UNIX environment

– Offers open interfaces for UNIX-type apps and interactive z/OS users

• Two z/OS UNIX software elements:

– z/OS UNIX System Services (USS)

– z/OS UNIX Application Services

• Enhances functionality of system services for application development

• Interprets commands, called shell scripts, from users or from programs

– Requests MVS services in response to those commands

• Java SDK installation on z/OS

– After installing SDK, path should contain the Java binary directory

– Four variables needed when using Java (sample values below):

PATH=/usr/lpp/java/J7.0/bin

CLASSPATH=/usr/lpp/java/J7.0/lib/classes.zip

LIBPATH=/usr/lpp/java/J7.0/include

JAVA_HOME=/usr/lpp/java/J7.0/

21

Job Management

Run-time environment of a Java batch program on z/OS

– Compiled Java program, often in a jar file

– JCL and the job execution environment

– Java program executed via JCL as a batch job

• Java programs often use a lot of system resources

– Region size must accommodate heap and stack storage,

as well as LE and Java code, and LE internal control blocks

– Execution via BPXBATCH, BPXBATSL or JZOS

– Integration of legacy and modern applications using JCL

– Job execution management using JCL conditions

22

Using JCL for Java Batch Execution

• IBM recommends using JCL to run Java batch

– Can be done by using BPXBATCH, BPXBATSL or JZOS

• Why use JCL?

– Behaves like regular batch work, with JES and job scheduling support

– Supports batch security environment

– Can organize the job flow by combining Java program execution

with other JCL steps comprising a business process

– Simplifies getting guaranteed CPU resources via WLM

• And getting memory for the JVM

– Can direct output to the spool for further handling or archiving

• Particularly with JZOS

23

//STDENV DD *

PATH=/bin

JAVA_HOME=/usr/lpp/java/J7.0

/*

Running Java Program as Batch Job• BPXBATCH supports running a Java batch job using JCL

– Utility program provided by IBM

– Runs shell scripts and executable files in the HFS* from JCL scripts

• Takes either SH or PGM as a first parameter

– When SH is specified, it expects the name of a shell script to run

– When PGM is specified, it expects the name of an executable file in the HFS

• BPXBATCH DD cards: STDOUT, STDIN, STDERR, STDENV

– STDENV is a properties file

containing variable=value pairs

• It can be set in line in the JCL

• Alternatively, it can be set via a readonly text file in HFS

//STDENV DD PATH=’/u/user1/env.profile’,PATHOPTS=READONLY

• It can also be set in a sequential dataset or a PDS

//STDENV DD DSN=SOMEUSER.ENVT.PROFILE,DISP=SHR

24* HFS – Hierarchical File System – also known as zFS

BPXBATCH – Sample JCL Stream

25

//JAVAPGM JOB (ACCT),'SOMEUSER',REGION=30M,

// CLASS=A,MSGCLASS=X,NOTIFY=&SYSUID

//* ----------------------------------------------------

//* Run Java program

//* ----------------------------------------------------

//RUN EXEC PGM=BPXBATCH,

// PARM='SH java -cp /u/someuser/samples MyPgm'

//SYSPRINT DD SYSOUT=*

//SYSOUT DD SYSOUT=*

//STDOUT DD PATH='/u/someuser/java.stdout',

// PATHOPTS=(OWRONLY,OCREAT,OTRUNC),

// PATHMODE=SIRWXU

//STDERR DD PATH='/u/someuser/java.stderr',

// PATHOPTS=(OWRONLY,OCREAT,OTRUNC),

// PATHMODE=SIRWXU

//STDENV DD DUMMY

//* ----------------------------------------------------

//* Copy the output of Java program

//* ----------------------------------------------------

//COPYOUT EXEC PGM=IKJEFT1B,DYNAMNBR=300,COND=EVEN

//SYSTSPRT DD SYSOUT=*

//HFSOUT DD PATH='/u/someuser/java.stdout'

//HFSERR DD PATH='/u/someuser/java.stderr'

//STDOUTL DD SYSOUT=*,DCB=(RECFM=VB,LRECL=133,BLKSIZE=137)

//STDERRL DD SYSOUT=*,DCB=(RECFM=VB,LRECL=133,BLKSIZE=137)

//SYSPRINT DD SYSOUT=*

//SYSTSIN DD DATA,DLM='/>'

ocopy indd(HFSOUT) outdd(STDOUTL)

ocopy indd(HFSERR) outdd(STDERRL)

/>

BPXBATCH execution

runs the java app called MyPgm

using 5 DD statements

Program output

is copied from files under USS

to SYSOUT=*

BPXBATCH and BPXBATSL

• BPXBATCH has limitations:

– Unable to handle standard I/O streams when run via JCL

– Executing programs cannot access DDs defined in enclosing JCL

• Runs specified shell scripts or executable file in a separate address space

– Consequently, DD cards are not visible to invoked programs

• BPXBATSL is an alternative utility program

– Addresses the limitations of BPXBATCH

– Provides an alternate entry point into BPXBATCH

– Forces a program to run using local spawn instead of fork/exec

• Therefore, it runs in same address space, making DD cards

specified in a JCL script available in the invoked program

26

JZOS Introduction

Why JZOS?

• Advantages of using JZOS

– Full support for DD statements

• E.g., allocating a temporary data set to

pass output from one step to a later step

– Directs stdout and stderr streams to standard MVS data sets

and JES SYSOUT data set

• Allows SDSF to be used to view output from Java programs

• With BPXBATCH, it is only possible to write/read to files in USS

– Seamless integration of Java programs with other job steps

– Simplifies console communication from Java applications

28

Job Management Using JZOS

• Integrate Java applications as part of a traditional MVS batch job

– Applications can be integrated seamlessly with other job steps

and MVS utilities using the JZOS job launcher

• The next several slides contain samples of JCL, shell script, and Java code

1. The JCL integrates Java with traditional batch job steps

• In this job, data is produced by a REXX exec called GENRCDS

• Output of exec is stored in a temporary data set called REXXOUT

• The data is sorted using the MVS SORT utility

• Output is stored in a temporary data set (SORTDATA)

• Sorted output is then input to Java program that updates a VSAM file

2. Shell script configures Java runtime environment

• Sets IBM_JAVA_OPTIONS (IBM Java SDK option) environment variable

• Options for performance tuning and remote debugging can be specified here

3. Java program that performs I/O using values set via DD statements

29

JZOS Job Sample (Page 1 of 2)

30

//JZOSXMPL JOB (ACCT),'ID123',REGION=300M,

// CLASS=A,MSGCLASS=X,NOTIFY=&SYSUID

//PROCLIB JCLLIB ORDER=THEMIS.JZOS.SAMPJCL

//**********************************************************

//* EXECUTE REXX EXEC TO WRITE DATA TO A TEMPORARY DATA SET

//* CALLED REXXOUT.

//**********************************************************

//GENREC EXEC PGM=IRXJCL,PARM='GENRCDS'

//SYSTSIN DD SYSOUT=*

//SYSTSPRT DD SYSOUT=*

//OUTFILE DD DSN=&&REXXOUT,DISP=(NEW,PASS),SPACE=(TRK,(2,1))

//SYSEXEC DD DSN=THEMIS.EXECS,DISP=SHR

//**********************************************************

//* EXECUTE SORT COMMAND AGAINST TEMP DATA SET (REXXOUT)

//* AND WRITE RESULTS TO ANOTHER TEMP DATA SET (SORTDATA)

//**********************************************************

//SORTSTEP EXEC PGM=SORT

//SYSIN DD *

SORT FIELDS=(1,80,CH,A)

/*

//SYSOUT DD SYSOUT=*

//SORTIN DD DSN=&&REXXOUT,DISP=(OLD,DELETE)

//SORTOUT DD DSN=&&SORTDATA,DISP=(NEW,PASS),SPACE=(TRK,(2,1))

/*

Rexx exec writes the temporary file

&&REXXOUT which is input to sort

External sort takes &&REXXOUT as

input and produces &&SORTDATA

JZOS Job Sample (Page 2 of 2)

31

//********************************************************

//* CALL JAVA PROGRAM USING JZOS TO READ FROM TEMP DATA

//* SET (SORTDATA) AND WRITE TO A PERMANENT FILE

//********************************************************

//JAVA EXEC PROC=EXJZOSVM,JAVACLS='ProcessRecords'

//INDATA DD DSN=&&SORTDATA,DISP=(OLD,DELETE)

//OUTDATA DD DSN=THEMIS.DATA,DISP=SHR

//STDOUT DD SYSOUT=*

//STDERR DD SYSOUT=*

//STDENV DD DSN=THEMIS.JAVAENV(JPROPS70),DISP=SHR

//********************************************************

//* DELETE THE BACKUP COPY FROM THE PREVIOUS RUN.

//********************************************************

//DELETE EXEC PGM=IDCAMS,REGION=1M,COND=(7,LT)

//SYSPRINT DD SYSOUT=*

//SYSIN DD *

DELETE THEMIS.DATA.BACKUP

SET MAXCC=0

/*

//********************************************************

//* MAKE BACKUP COPY OF RESULTS IF PREVIOUS STEPS SUCCEED.

//********************************************************

//BACKUP EXEC PGM=IEBGENER,COND=(7,LT)

//SYSPRINT DD SYSOUT=*

//SYSIN DD DUMMY

//SYSUT1 DD DSN=THEMIS.DATA,DISP=SHR

//SYSUT2 DD DSN=THEMIS.DATA.BACKUP,DISP=(NEW,CATLG),SPACE=(TRK,(2,1))

Java program, ProcessRecords,

uses &&SORTDATA via the INDATA DD

to update a VSAM dataset

Final two steps manage backup

copies of the updated VSAM data

Shell Script to ConfigureJava Runtime Environment

32

# This is a shell script which configures

# environment variables for the Java JVM 7.0.

# Variables must be exported to be seen by launcher.

. /etc/profile

export JZOS_HOME=/usr/lpp/java/J7.0

export JAVA_HOME=/usr/lpp/java/J7.0

export PATH=/bin:"${JAVA_HOME}"/bin:

LIBPATH=/lib:/usr/lib:"${JAVA_HOME}"/bin

LIBPATH="$LIBPATH":"${JAVA_HOME}"/bin/classic

LIBPATH="$LIBPATH":"${JZOS_HOME}"

export LIBPATH="$LIBPATH":

# Customize your CLASSPATH here

CLASSPATH=/u/themis/examples

# Add JZOS required jars to end of CLASSPATH

for i in "${JZOS_HOME}"/*.jar; do

CLASSPATH="$CLASSPATH":"$i"

done

export CLASSPATH="$CLASSPATH":

# Additional JZOS specific options may be included, e.g. to

# specify character encoding for DD STDOUT and STDERR; to supply

# additional arguments to main; to configure JVM options; etc.

JAVA_HOME and JZOS_HOME

are both set and added to LIBPATH

CLASSPATH is set to include

directory containing Java program

as well as jar files used by JZOS

Java Program Performing I/O

Using DD statements (Page 1 of 2)

33

import com.ibm.jzos.*;

public class ProcessRecords {

public static void main(String args[]) {

ZFile inFile = null, outFile = null;

byte[] buffer = null;

int nRead;

try {

// References to input/output data sets via DD statements.

inFile= new ZFile("//DD:INDATA", "rb,type=record,noseek");

outFile = new ZFile("//DD:OUTDATA", "wb,type=record,noseek");

// Allocate buffer depending on record length of input data set

buffer = new byte[inFile.getLrecl()];

String enc = ZUtil.getDefaultPlatformEncoding();

// Read records as long as more records available

while ((nRead = inFile.read(buffer)) >= 0) {

// Construct string based on record read and default platform encoding

String line = new String(buffer, 0, nRead, enc);

System.out.println(line);

// Write to the specified output data set

outFile.write(buffer);

outFile.flush();

}

Java program uses ZFile objects

to read input and write output based on files

attached to specified DD names

ZFile object read method is

passed a buffer used to load data into and

returns the number of bytes read

Input data in buffer is used to

create a String with the correct encoding

and then write it to output file

More to come on ZFile

Java Program Performing I/O

Using DD statements (Page 2 of 2)

34

} catch (Exception e) {

// Prints stack trace of exception to standard error output

e.printStackTrace();

System.exit(8); // Important to quit with the right RC

} finally { // Close input file

if (inFile != null) {

try {

inFile.close();

} catch (Exception e) {

e.printStackTrace();

System.exit(8);

}

} // Close output file

if (outFile != null) {

try {

outFile.close();

} catch (Exception e) {

e.printStackTrace();

System.exit(8);

}}

}

System.out.println("Done!");

System.exit(0); // Important to quit with correct RC

}

}

Really just exception handling

and file management happening on this page,

and specifying returning code

Console Communication and

Logging in Java on z/OS

Console Communication with Batch Jobs

• Common practice to use Write-To-Operator (WTO) macro

– Running programs (jobs) write messages to job log and system console

– Typically used to report program status

• E.g., record events when failure occurs in long-running job

• JZOS provides APIs to support these capabilities in Java apps

– MvsConsole class and WtoConstants interface

• System Display and Search Facility (SDSF)

– Communicate with running jobs using MVS commands

– Also, interface to system log

• View SYSLOG by typing log command in SDSF

– Log includes all information from all jobs running on the system

36

Using log4j in the Z/OS Environment

• log4j: popular open source project from Apache Software Foundation

– Generic framework for a light-weight logging system for Java apps

– Allows developer to control which log statements are output to where

– Fully configurable at runtime using external configuration files

– Can use log4j to log messages at different levels including

debug, info, warn, error, and fatal.

– Can also direct output of messages to different locations

• Depending on the severity of the messages

• For more information about log4j: http://logging.apache.org/

37

Accessing MVS

Data Sets With JZOS

MVS Data Sets

• Enterprise data is stored in files and relational databases

– Java Database Connectivity (JDBC) supports access to relational data

– JDBC APIs can be used in a stand-alone environment

• Coverage of JDBC is not part of today’s discussion

– Libraries with classes and java beans are available for file access

• A Java application running in the UNIX environment

may require access to MVS data sets

– No issue in using Java to access files stored in HFS format in UNIX

• How does Java access these MVS data sets?

– The JZOS toolkit library

39

JZOS Toolkit Library

• JZOS toolkit library supports access to mainframe file systems

that are not available via the standard Java I/O library

– JZOS provides thin wrappers for z/OS C/C++ Library functions

to access MVS data sets

• JZOS allows Java programs to access any MVS data sets

supported by the C/C++ library, including:

– Sequential files

– Partitioned Data Set (PDS) and Partitioned Data Set Extended (PDSE)

– Virtual Sequential Access Method (VSAM) KSDS, RRDS, and ESDS

40

JZOS Sample Code

• The following example uses JZOS to create a copy of all members of a data set

– Creates a list of PDS members

– BufferedReaders and BufferedWriters wrap the JZOS FileFactory class

• Read data from each member into the new member using text stream processing

• Program invoked with an argument: java MemberCopy USER123.MYDSN

– The argument specifies the data set to be copied

– Members are copied into USER123.MYDSN.COPY

– The program prints out the new members as they are copied

• It also prints the number of lines copied and the total members copied

• PdsDirectory and PdsDirectory.MemberInfo are used by the program

– It generates an ArrayList of Strings containing the names of all of the members

in the source data set and the corresponding members in the copy data set

– Steps through the ArrayList by two by two

• Uses a BufferedReader to read from the source data set member

• Uses a BufferedWriter create a copy of it

41

Copying Data Set Members (page 1 of 2)// imports omitted

public class MemberCopy {

public static void main(String args[]) {

// check for existence of args[0] omitted

try {

copyMembers(memberList(args[0]));

} catch (Exception e) {

// error handling omitted

}

}

public static ArrayList memberList(String inDsn) {

ArrayList members = new ArrayList();

try {

// inDsn preprocessing omitted

String outDsn = inDsn + ".COPY";

// check for existence of input dsn and non-existence of output dsn omitted

PdsDirectory dir = new PdsDirectory("//" + inDsn);

for (Iterator iter = dir.iterator(); iter.hasNext();) {

PdsDirectory.MemberInfo info = (PdsDirectory.MemberInfo) iter.next();

String fullInDsn = inDsn + "(" + info.getName() + ")";

String fullOutDsn = outDsn + "(" + info.getName() + ")";

members.add(fullInDsn);

members.add(fullOutDsn); }

} catch (IOException io) { io.printStackTrace(); }

return members;

}

42

Call memberList to create

List of members and pass

that List to copyMembers

Open the input PDS and

Iterate through members

Add the PDS qualified

member name to the List for

both from and to DSNs

This class copies all members of a PDS to another PDS

Copying Data Set Members (page 2 of 2)public static void copyMembers(ArrayList members) {

BufferedReader rdr = null;

BufferedWriter wtr = null;

int count = 0;

try {

for (int i = 0; i < members.size(); i += 2) {

String input = ZFile.getSlashSlashQuotedDSN((String)members.get(i));

String output = ZFile.getSlashSlashQuotedDSN((String)members.get(i + 1));

rdr = FileFactory.newBufferedReader(input);

wtr = FileFactory.newBufferedWriter(output);

String line;

long llines = 0;

while ((line = rdr.readLine()) != null) {

wtr.write(line);

wtr.write("\n");

llines++;

}

count++;

System.out.println("Copied " + input + " lines=" + llines);

if (wtr != null) wtr.close();

if (rdr != null) rdr.close();

}

} catch (IOException io) { io.printStackTrace(); }

System.out.println(count + " members copied to PDS");}

}

43

Use BufferedReader and

BufferedWriter to do copy List contains each member in

“from PDS” followed by the corresponding

member in the “to PDS”

Static ZFile

method used to

correctly format

qualified member

nameStatic FileFactory methods used to

open source and destination members as a

BufferedReader and a BufferedWriter

Line by line copy

BPXWDYN• BPXWDYN is a callable service that provides an interface to support

dynamic allocation and dynamic output services

– Supports data set allocation, unallocation, concatenation, retrieval of certain

allocation information, and addition and deletion of output descriptors

– Has been available to be called from REXX, C, Assembler, and PL/I

– Through JZOS, it can be now also be called from Java

• Use ZFile's allocDummyDDName() and bpxwdyn() together

– When JZOS files are opened by dataset name (as opposed to DD name) they are always opened for exclusive use (DISP=OLD)

– Often desirable to have non-exclusive dataset access (DISP=SHR)

– Using these two static ZFile methods, a dummy DD can be allocated, then

used in a call to BPXWDYN to read the underlying dataset non-exclusively

44

public ZFile openShared(String filename) throws IOException {

String ddname = ZFile.allocDummyDDName();

ZFile.bpxwdyn("alloc dd("+ddname+") dsn("+filename+") shr");

return new ZFile("//DD:" + ddname, "rb,type=record,noseek");

}

Working With VSAM Data Sets

• Types of VSAM data sets:

– Key-Sequenced Data Set (KSDS) – Entry-Sequenced Data Set (ESDS)

– Relative Record Data Set (RRDS) – Linear Data Set (LDS)

• JZOS provides support for all VSAM data sets except LDS

• VSAM terminology:

– Logical record: unit of data stored in a VSAM data set

• Contains fields of data and a key, used to retrieve a specific record

– Cluster: set of related components comprising a VSAM data set

• VSAM data sets are referred to by their cluster names

• Contains a data component and an index component

– IDCAMS: utility program used to define VSAM data sets

• Data sets must be allocated before Java applications can work with them

– JZOS does not support allocation of VSAM data sets

45

Accessing VSAM Data Set with JZOS

46

• JZOS provides several methods to manipulate VSAM data sets

• ZFile has methods for adding, deleting and updating records

in a VSAM data set

– Also methods to get information such as logical record length (LRECL)

– ZFile supports reading a portion of a record (useful for large records)

• Read a record from a VSAM file:

ZFile zfile = new ZFile(filename, options);

zfile.locate(key, offset, keyLen, ZFile.LOCATE_KEY_EQ));

int nRead = zfile.read(recBuf);

– File is opened; a record is located by matching its key using the offset

to the start of key buffer; record is then read into byte array, recBuf

– Records must be read prior to write, update and delete operations

VSAM Access Code Snippet (page 1 of 2)String filename = args[0];

String options = "ab+,type=record";

ZFile zfile = new ZFile(filename, options);

int lrecl = zfile.getLrecl();

int keyLen = 10;

// construct some records with key prefixes

byte[] rec_1 = padToLength("AAAAAAAAAARecord 1", lrecl)

.getBytes(ZFile.DEFAULT_EBCDIC_CODE_PAGE);

byte[] rec_2 = padToLength("BBBBBBBBBBRecord 2", lrecl)

.getBytes(ZFile.DEFAULT_EBCDIC_CODE_PAGE);

byte[] rec_3 = padToLength("CCCCCCCCCCRecord 3", lrecl)

.getBytes(ZFile.DEFAULT_EBCDIC_CODE_PAGE);

byte[] recBuf = new byte[lrecl];

int nRead, nUpdated;

// write the three records

zfile.write(rec_1);

zfile.write(rec_2, 0, rec_2.length); // alternate form

zfile.write(rec_3);

// point to the first record

zfile.locate(rec_1, 0, keyLen, ZFile.LOCATE_KEY_EQ);

// read back the record and verify its contents

nRead = zfile.read(recBuf);

System.out.println("Record 1 \t\t:" + printBytes(recBuf,nRead));

47

File open options ab+ support

reading, writing and updating

ZFile constructor

returns ZFile instance

Write each of the 3 records

out to the VSAM Cluster

Position on record one by

matching on its key value

The ZFile read method places

record data into byte buffer passed to it;

returns number of bytes read

The padToLength method

simply appends as many @ chars

as are needed to reach the length

specified in the 2nd argument

Three byte arrays containing

test records with 10 byte keys

printBytes decodes the byte array

using EBCDIC character encoding

VSAM Access Code Snippet (page 2 of 2)

// update a record

byte[] rec_1U = padToLength("AAAAAAAAAARecord 1 updated", lrecl)

.getBytes(ZFile.DEFAULT_EBCDIC_CODE_PAGE);

nUpdated = zfile.update(rec_1U);

// read back the record and verify its contents

zfile.locate(rec_1, 0, keyLen, ZFile.LOCATE_KEY_EQ);

nRead = zfile.read(recBuf);

System.out.println("Record 1 After Update \t:" + printBytes(recBuf,nRead));

// update second record, using alternate update() form

byte[] rec_2U = padToLength("BBBBBBBBBBRecord 2 updated",

lrecl).getBytes(ZFile.DEFAULT_EBCDIC_CODE_PAGE);

nUpdated = zfile.update(rec_2U, 0, rec_2U.length);

// read back a portion of the record

byte[] partialRecBuff = new byte[10];

nRead = zfile.read(partialRecBuff, 0, 10);

System.out.println("Partial Record " + nRead + "\t:" +

printBytes(partialRecBuff, partialRecBuff.length));

// delete a record

zfile.locate(rec_1, 0, keyLen, ZFile.LOCATE_KEY_EQ);

nRead = zfile.read(recBuf);

zfile.delrec();

48

ZFile update method uses record

data in byte buffer to update current record;

returns number of bytes updated

This version of update takes the starting

position and length along with byte array

This version of read includes a

starting position and length to read only part of the

record data into the provided byte buffer

Creating Java Record Classes

From COBOL Copybooks

Creating Java Record Classes • z/OS datasets often contain records described by COBOL copybooks

– Contain fields that conform to the set of valid COBOL types

• Alpha-numeric, unscaled numeric, packed decimal, binary, etc.

– A Java program may need to process this kind of dataset

• May be difficult to convert from COBOL to Java data types

• JZOS offers a complete set of field converter types *

– Operate on a byte array

– Convert from COBOL types to Java types

• IBM provides a sample class to illustrate how a Java class can be hand coded

to provide access to COBOL described record fields from a Java program

– Illustrates the difficulty in hand coding these records

– Especially with more complicated COBOL copybooks

• E.g. using features like Occurs Depending On (ODO)

50* Contained in com.ibm.jzos.fields package

Automating Generation

of Java Record Classes • Can build Java record class programatically from a copybook

– Automated process uses input from the COBOL copybook

• JZOS includes a development time tool for doing this

– com.ibm.jzos.recgen.cobol.RecordClassGenerator

– Uses the ADATA output of z/OS Enterprise COBOL compiler

– Generates a Java class to map a selected copybook record

• Use ant to create Java record classes to support these datasets

– Submits a job to generate an ADATA file for a COBOL copybook

• Machine readable file describing structure and content of compilation

– Downloads the file and runs RecordClassGenerator

• Uses the ADATA file to generate the associated Java class

• Generated Java record classes can be used like any other Java class

51

Customizing Generated Record Classes • Modifying generated class to add functionality not recommended

– Subclassing is the preferred option

– By default, generated record fields are protected

• Accessible from the subclass

– Subclassing also provides the prospect of implementing related interfaces

– May enable generated record classes to be used in various implementations

52

package com.jzos.sample.domain.records;

public class MyClassImpl extends MyClass implements MyInterface {

static {

COMMENTS.setTrim(true);

NAME.setTrim(true);

}

...

public Timestamp getTimestamp() {

return TimestampUtil.getTimestamp(getTimestampDate(), getTimestampHhmmss());

}

public void setTimestamp(Timestamp timestamp) {

setTimestampDate(TimestampUtil.getYyyyxxxx(timestamp));

setTimestampHhmmss(TimestampUtil.getHhmmss(timestamp));

}

}

Class generated with stringTrim false.

This static block overrides string trimming

behavior for those two fieldsNo defined COBOL timestamp datatype.

These getter and setter methods use a helper class to

encode/decode java.sql.Timestamp instances

Remote Debugging of

Batch Java Applications

Debugging on z/OS is Remote

54

• There is no Java IDE (e.g. Eclipse) running on z/OS

– Thus, remote debugging is of particular importance when a Java

program accesses resources that are only accessible on z/OS

• Sequential data sets, VSAM files, etc.

– Also extremely useful for problems that occur only on z/OS

• Even when the Java program does not access any z/OS specific resources

• Eclipse Java debugger is designed to use client/server architecture

– Allows launching the stand-alone Java application on z/OS and

debugging it from the workstation-based Eclipse workbench

Remote Debugging• To debug a Java application running on z/OS remotely from

Eclipse, the program is launched on z/OS in debug mode

so that JVM is waiting for a connection from your debugger

• Overview of the basic steps:

1. Build the Java programs with necessary debug information

• Controlled in Eclipse by configuring preferences

2. Deploy Java application to target z/OS run-time environment

• Copy jar files to USS directories; configure environment variables

such as CLASSPATH and LIBPATH; prepare shell scripts and JCL

3. Start Java application on z/OS using appropriate VM arguments

to specify debug mode and communication port for debugger

4. Start debugger using a remote debugging configuration and

specify address and port of the target machine running z/OS

55

Using the Eclipse Debugger• Before starting debugger, set a breakpoint in your Java code

– Later on, debugger locates source code. Both source code and

breakpoint show up in source code view of debug perspective

• Click Debug

– The launcher attempts to connect to JVM at specified address and port

– Debug view shows that debugger connects to the JVM at host/port

• E.g. 10.1.11.24:8000, with main thread suspended at the breakpoint

• Step through program as if it is running on the local machine

• Debugger context menu supports disconnect, terminate, relaunch

– Both disconnect and terminate will disconnect debugger from remote JVM

• Disconnect does not terminate the remote JVM, but terminate does

– If “Allow termination of remote VM” was checked in the configuration

56

Eclipse Debugger

• During debugging you can

– Watch and inspect variable values using Variables view

• Arguments passed to Java program are shown in this view

– Manipulate variables in Variables view

• Change values of primitives and Strings

– Add new breakpoints; resume running; continue to step through code

• Output of System.out.println() is not visible in Eclipse console

– If program is launched in USS, output will be on your USS session screen

– If program is launched using BPXBATCH, it can be found either in a USS file

or in SYSOUT, depending on JCL/Proc

• Eclipse debugger can be used for nearly all of the the functions

it provides to debug a Java program running locally

57

Local Debugging

• The Java application may be debugged locally in Eclipse

– If it accesses only zFS files and z/OS-based DB2 and WebSphere MQ

• Local files can be used for debugging instead of zFS files

• A JDBC Type 4 driver or DB2 Connect can be set up for

remote DB2 access

• A WebSphere MQ client can be set up to access queues in an

MQ queue manager on z/OS

• Local debugging is simpler than remote debugging

– No need to manually start a JVM before starting Eclipse debugger

– Eclipse workbench console supports standard I/O

58

Exploiting zIIP

zIIP Overview• IBM System z Integrated Information Processor (zIIP)

– Low priced specialized processing unit

• IBM does not impose software charges on zIIP capacity

– Provides economical Java execution environment

– Exploits traditional advantages of the zSeries platform

• Quality of service and integration

• zIIP processors cannot perform all tasks

– Can only assist general purpose CPs

• IBM JVM processing cycles can be executed on installed zIIPs

– No modification to Java applications is required

• zIIPs can help increase general purpose processor productivity

– May contribute to lower overall computing cost for Java on System z

– Amount of general purpose processor savings varies

• Based on amount of Java code executed by zIIPs

60

zIIP Can Reduce Execution Cost

• System z Integrated Information Processor (zIIP) reduces costs

– Previously called System z Application Assist Processor (zAAP)

• Various APIs that are typically used in Java batch applications

give potential zIIP utilization including pure Java calculations

– Bitwise operations

– Java for zOS (JZOS) library

– JDBC drivers for DB2 on z/OS

• Estimate of potential zIIP utilization of various applications

– Potential benefits of zIIP more clearly evaluated by estimating

various APIs used in a new or existing mainframe application

• IBM experimental results show much potential

– Java batch apps can help manage total cost of mainframe applications

61

zIIP Utilization Potentials• zIIP utilization estimate and projection tool

• At one extreme situation - 99% of application is written natively

– Application is invoked via Java Native Interface (JNI)

– For such applications, the benefit of zIIPs is very limited

• zIIPs are not utilized at all

• At the other extreme, application is written purely in Java

– Can expect a very high zIIP utilization to run the application

• Estimating zIIP utilization is usually less straightforward

– Typical applications fall between these two extremes

• IBM measured several known workloads to estimate

ratio of Java execution time to total system CPU time

– Range of results depending on type of application

– Typically around 40 – 60%

62

Final Thoughts and Database Impact• zIIP usage provides incentives to develop new batch Java applications

or re-write existing batch applications using Java

– Particularly if the applications have lots of data set I/O

• Using JZOS, batch jobs run completely on zIIP processors

– Frees usage of main CPs for other tasks

• Takes advantage of lower price of zIIP processors

• Reduces overall cost of mainframe apps

– Modernizes existing applications by moving them to Java

• Easier to find Java developers to manage mainframe applications

• JDBC specifies four types of DB2 database drivers

– Type 2 and Type 4 drivers are most commonly used in the enterprise today

– Type 2 drivers are written partly in Java – the rest in native code

• They use a data source specific native client library

• zIIP utilization below 50%

– Type 4 drivers are written purely in Java

• Implement network protocol for a specific data source

• 100% zIIP utilization

63

Java Native Interface

(JNI) Introduction

Java Native Interface (JNI)

• Some situations prevent application being written entirely in Java

– In such cases, Java Native Interface (JNI) might provide a solution

• JNI can be used to:

– Invoke native functions from a Java application

– Call Java methods from a native application

• JNI was originally designed to be combined with C/C++

– Can now be used to combine Java with COBOL

• Available in Enterprise COBOL for z/OS V3R4

• Relatively new capability

• Can use C/C++ to access z/OS UNIX Assembler Callable Services

– Enables access to resources that are inaccessible directly through Java code

– Also possible to use some C/C++ code to link to an Assembler module

• Allows use of JNI to integrate Assembler into a Java application

65

JNI Pros and ConsAdvantages:

• Enables reuse of existing code written in another language

• Performance of native code is generally faster than Java code

– Some performance gains are therefore possible

Disadvantages:

• Using JNI in a J2EE environment such as WebSphere Application

Server may crash the server if errors occur in the native code

• zIIP processors are available exclusively to run Java code

– To maximize the benefits of zIIPs, minimize the use of native code

66

Calling a COBOL Program from Java

• A Java program can be invoked via command prompt under USS

or it can be submitted as a batch job

• Since JNI only enables Java programs to call C/C++ programs,

a glue layer is required to invoke the COBOL program

– C glue layer program can call COBOL programs via z/OS Language

Environment (LE) Inter-Language Communication (ILC) mechanism

• LE enables interlanguage communication among C, C++, COBOL, and PL/1

• Allows calling any existing program regardless of implementation language

• Java application uses a C function to invoke COBOL program

• Details are beyond the scope of this discussion

67

Where Do I Go

From Here?

Developing Java Applications for z/OS

Batch Execution Course Information

http://www.themisinc.com/PDF/JA1080.pdf

Learning More About Java,

OO and Database Interaction

• Java 1 (Intro Java) and Java 2 (Intermediate Java)

– These courses spend two weeks in Java Standard Edition

– Both courses are necessary to get a complete background

• Java Enterprise – Servlets and JSPs

– One week course on these critical components of Java EE

• Object Oriented Analysis and Design

– This course gives the budding OO developer a solid

background in OO requirements specification and UML

• Many other courses depending on your environment

– Enterprise Java Beans (EJBs); Java Persistence API (JPA);

Java Messaging Service (JMS); Java Server Faces (JSF);

Web Services with Java; Design Patterns; Spring;

XML; HTML; Hibernate; Struts; many more70

http://www.themisinc.com/Course.aspx?category=600

Thank you

for coming

US 1-800-756-3000

Intl. 1-908-233-8900

www.themisinc.com

On-site and Public

Instructor-led

Hands-on

Training

Over 400 IT Courses

Customization Available