iam concepts & facilities guide · 2020. 7. 10. · the iam/rls feature provides record- level...

1

IAM Concepts & Facilities Guide

The Mainframe Software Partner for the Next 50 Years

2

Contents IAM .........................................................................................................................................1

Introduction ............................................................................................................................5

What Is IAM? .......................................................................................................................5

PART 1 – Overview of VSAM ....................................................................................................6

Introduction ........................................................................................................................6

Defining a VSAM File ...........................................................................................................7

The VSAM File Structure ......................................................................................................7

Loading the Data .................................................................................................................8

CI% FREESPACE ....................................................................................................................8

Adding Records to the File ...................................................................................................9

CA% FREESPACE ................................................................................................................10

Inserting ‘HIGH KEY’ Records .............................................................................................12

DASD Utilization ................................................................................................................13

Using Buffers .....................................................................................................................13

PART 2 – VSAM Problems ......................................................................................................15

Introduction ......................................................................................................................15

CI/CA Splitting ...................................................................................................................15

DASD Usage .......................................................................................................................17

Buffer Usage ......................................................................................................................18

Record-Level File Sharing ...................................................................................................20

Summary ...........................................................................................................................20

PART 3 – IAM ........................................................................................................................21

Introduction ......................................................................................................................21

IAM: The Power over VSAM ..............................................................................................21


3

The VSAM Interface (VIF)...................................................................................................23

Creating an IAM File: IDCAMS Support ..............................................................................23

IAM Overrides ...................................................................................................................25

The IAM File Structure .......................................................................................................25

The IAM “PRO” File Structure ............................................................................................27

File Reorganization ............................................................................................................28

Transporting IAM Files .......................................................................................................28

Data Compression .............................................................................................................28

Space Release ....................................................................................................................29

Support for PAV .................................................................................................................30

Additional Performance Features ......................................................................................30

IAM under CICS .................................................................................................................31

IAMINFO–Execution Time Statistics ...................................................................................32

Special Feature: Record-Level Sharing ...............................................................................33

IAM Record-Level Sharing–Summary .................................................................................33

IAM Record-Level Sharing–Single System (IAM/RLS) ..........................................................33

IAM Record-Level Sharing–On a SYSPLEX (IAM/PLEX) ........................................................36

Journaling & Recovery .......................................................................................................37

Special Feature: Alternate Index (AIX) Support ..................................................................38

ESDS Support .....................................................................................................................40

Identifying Candidates for IAM Conversion ........................................................................41

IAMSMFVS.........................................................................................................................41

SMS Support ......................................................................................................................43

IAM Reliability ...................................................................................................................43

IAM Installation .................................................................................................................43

IAM ISPF Panels .................................................................................................................44


4

SUPERC Support ................................................................................................................45

Defining an IAM File with the ISPF Panels ..........................................................................45

Summary ...........................................................................................................................46

PART 4 – File Reorganization .................................................................................................47

Introduction ......................................................................................................................47

IAMDRCC – Dynamic Reorganization .................................................................................48

IAMDRCC – CICS Customization Requirements ..................................................................50

IAMSRORG – IAM Special Reorganization Process .............................................................51

PART 5 – Appendix ................................................................................................................53

IAM INNOVATION ACCESS METHOD ..................................................................................53

Customer Feedback ...........................................................................................................55

Benchmarks (IAM vs VSAM with SMB) ...............................................................................56


5

Introduction Each of the Compuware products is described in a range of Concepts & Facilities Guides. In this particular guide, we look at IAM.

What Is IAM?

IAM is a reliable high performance disk file manager that can be used in place of VSAM KSDS or VSAM ESDS data sets for batch, TSO and online processing.

Customers with the optional Alternate Index feature can use IAM for alternate indexes and the associated base clusters and use IAM for VSAM RRDS data sets.

IAM offers a level of performance and reduction in the use of computer system resources that provides substantial savings for most applications that utilize VSAM data sets. IAM drastically reduces DASD space requirements through maximizing the storage capabilities of each DASD device and a Data Compression feature that does not consume excessive CPU time.

IAM utilizes a sophisticated buffering mechanism, called Real Time Tuning that responds quickly to the I/O demands of application programs, resulting in substantial decreases in physical I/O.

The IAM/RLS feature provides record-level file sharing on IAM files, allowing multiple concurrent access for read and/or write processing by CICS transactions, batch programs and TSO users running on the same LPAR.

The additional-cost IAM/PLEX feature provides similar file-sharing capabilities across multiple LPARs in a z/OS Sysplex.

PART 1 and PART 2 give a basic overview of the fundamentals of the VSAM file structure and explain how this structure is the cause of most of VSAM’s performance problems.

Note: These first two sections are aimed at readers who are unfamiliar with VSAM and who would like to gain a better understanding of its design and structure. Readers who are already familiar with VSAM may want to go directly to PART 3.

PART 3 of this guide we look at how IAM can be used to provide a quick and relatively cheap solution to the problems caused by VSAM.

PART 4 looks at FDRREORG, a separate cost-option product that can be used to reorganize VSAM and IAM files and compress PDS’s.


6

PART 1 – Overview of VSAM This section provides a general overview of VSAM.

Introduction

IBM’s primary access method, VSAM, has been around now for several decades, yet it is still frequently used as the basis for high-profile ‘bought in’ packages—such as Financial and Personnel applications—company’s Management (e.g. Financial Directors) can often use these applications, as well as the regular workforce.

The design of VSAM directly affects the performance and efficiency of these systems. Common problems caused by VSAM are an increase in Response Times during the online day and/or an unacceptably long overnight Batch Window, which then impacts the availability of the online system the following morning. The use of DASD space can also be an issue, especially if users of an application are charged for their DASD consumption. Some files can grow very large—to several Gigabytes and more.

When these problems are being experienced, the users of the system (including Management) are unlikely to accept, or even understand, that the cause is due to the VSAM file structure. Equally, they find it unacceptable to be told that, because of the nature of VSAM, there is little that can be done without a great deal of manual intervention and tuning. They want quick solutions. Unfortunately, traditional ‘quick’ solutions, like hardware upgrades, can only give short-term relief and are often very expensive to implement.

In order to understand fully the problems inherent in VSAM, it is first necessary to take a look at the VSAM file structure, as it is here that the root cause of the problems exists. We will concentrate on the structure of VSAM KSDSs although, as you see in PART 3 – IAM, IAM also supports Alternate Index (AIX), RRDS and ESDS data sets. Readers already familiar with VSAM may wish to bypass the rest of PART ONE and move on to PART TWO. It is assumed that all readers have a basic understanding of Catalogs, VVDSs and VTOCs.


7

Defining a VSAM File

Let’s start by looking at how VSAM files are created. Under DFSMS, this can be done through regular JCL but, for the purposes of this overview, we assume that IDCAMS is still the preferred method. The following JCL extract illustrates a DEFINE of a single Index VSAM KSDS using IDCAMS:

The IDCAMS job on the previous page would result in a VSAM KSDS cluster being created with the name ‘MY.TEST.DATASET’. This cluster would be comprised of two components:

• The ‘DATA’ Component: Containing the data records for the file, stored in key sequence, hence the name Key Sequence Data Set (KSDS).

• The ‘INDEX’ Component: Containing records indexing the data component.

In our example job both components would be allocated on MVS001, the cluster would be cataloged to MVS001 and an entry placed in the VVDS on the volume for both the Cluster and the individual Component names. In addition, DSCBs for the component names would be placed in the VTOC on MVS001. It is not, however, this recording mechanism for VSAM which is the cause of its poor performance. Instead, it is the internal structure of the file itself.

The VSAM File Structure

Within the DATA and INDEX components, an internal structure is created at the time of the DEFINE. We will be concentrating on the DATA component, but a similar design (and resulting performance problems) can be experienced in the INDEX component as well.

The DATA component is divided into Control Areas (CAs) that are, in turn, divided into Control Intervals (CIs). A typical CA size is 1 CYLINDER and the CIs contained within each CA are of a size defined by the CISIZE parameter coded in the DEFINE. In our example earlier, if the file was allocated on a 3390, twelve CIs of 4096 bytes would fit onto a track and, as the 3390 has 15

//DEFVSAM EXEC PGM=IDCAMS //SYSPRINT DD SYSOUT=* //SYSIN DD * DEFINE CLUSTER (NAME(MY.TEST.DATASET) – VOLUMES(MVS001) - SPEED REUSE ) - DATA (RECORDS(5000 500) – FREESPACE(10 20) - KEYS(10 0) - RECORDSIZE(256 256) – CISIZE(4096)) /*


8

tracks per cylinder, the CA would contain 180 CIs. The internal structure of the DATA component would therefore look something like this:

Loading the Data

Once the file has been DEFINE’d, the data records must be loaded into it. Remember that the data is stored in a KSDS in key sequence order. The initial loading of data is usually done using an IDCAMS REPRO, as shown below. In the following example, the input data is coming from a previously created sequential file (SEQFILE) held on disk, and is going out to a VSAM file called ‘NEW.VSAM.FILE’ that has already been DEFINE’d in a previous step or job.

CI% FREESPACE

When the KSDS is DEFINE’d, a decision has to be made as to whether any freespace must be reserved in the file during the initial LOAD. If the KSDS is only used for READ purposes, there would not be any need to reserve freespace. However, if records are to be inserted into the file (or increased in size as a result of being updated), space must be reserved to allow these inserts or updates to take place.

The first type of freespace available in a VSAM KSDS is called CI% FREESPACE. This is reserved when the file is DEFINE’d, via the FREESPACE parameter. In our earlier IDCAMS example, we requested CI% Freespace of 10%. When REPRO loads each CI with data, 10% of the space would

//DEFVSAM EXEC PGM=IDCAMS //SYSPRINT DD SYSOUT=* //SEQFILE DD DSN=seq.file,DISP=SHR //SYSIN DD * REPRO INFILE(SEQFILE) OUTDATASET(NEW.VSAM.FILE) /*

CI 1 CI CI CI CI

CI CI CI CI CI 180

CA 1

CI 1 CI CI CI CI

CI CI CI CI CI 180

CI 1 CI CI CI CI

CI CI CI CI CI 180

CA 2

CA 3


9

be reserved for future inserts. After the load has completed, every CI in every CA throughout the whole KSDS would look something like this:

Adding Records to the File

As additional records are subsequently added to the KSDS after the initial load, VSAM attempts to place them into the appropriate CI. In the example below, a record with the key of ‘2’ is to be added to the KSDS. To maintain the key sequence, it must be placed in the CI, which already contains records with keys of ‘1’ and ‘3’.

CI

10% Reserved CI Freespace

3

3

1 2 4 5 8 10

12 1 2 4 5 8 1

0 12


10

CA% FREESPACE

Depending on the level of insert activity and the amount of freespace coded on the DEFINE, it may be possible to exhaust the available freespace in a CI, such that additional inserted records cannot fit into the correct CI.

To cope with this, another type of freespace, called CA% FREESPACE, is also reserved in the file. This is achieved using the second value on the FREESPACE parameter in the IDCAMS DEFINE and it involves reserving whole free CI’s within a CA during the load process:

When a CI has exhausted its CI% Freespace, it is divided into two in an operation known as a ‘CI Split’.

Free CI Free CI

6 ?

1 2 3 5 7 9 10

11

12

14


11

Approximately half of the records from the full CI are moved into one of the reserved free CI’s at the end of the same CA (see below). Once the CI Split has completed, the new record that caused the split to occur can then be inserted into the original CI.

Unfortunately, CA% freespace can also become exhausted. If, for example, a large number of records are being added to a file and they all have a similar key value, it is quite common for all of the free CI’s in a CA to become used as a result of the CI Splitting (see below). How then can additional records be added?

CI 1 2 4 6 7 8 10 11 13 15 16 19

1 2 4 6 7 8

10 11 13 15 16 19


12

Once all free CI’s in a CA have been exhausted, new CI’s cannot be used from another CA. Instead, VSAM has to do what is known as a ‘CA split’:

In the ‘CA split’ process, a new CA is formatted at the end of the file to include the same CI and CA freespace as used throughout the rest of the file. Once the new CA has been formatted, half of the CIs in the original CA are moved into the new one (see above).

Unfortunately, it doesn’t end there. Once the CA split has completed, a CI split still has to occur in the original CA to allow the original CI to take the inserted record! Only when the CI split has completed successfully can the record finally be added to the file.

PART TWO of this Concepts Guide highlights the effects that this CI/CA space can have on the performance of the KSDS, both during the split and also when the records are later being processed. It also discusses the inadequacy of the two commonly used alternatives for reducing the number/effect of splits–namely ‘Increasing freespace’ and ‘Running regular REORGs’.

PART THREE of this Guide then shows how the IAM file structure handles inserted records with a much greater level of efficiency.

Inserting ‘HIGH KEY’ Records

So far, we have concentrated on inserting records into the file which have keys lower than the ‘last’ record in the file when it was first loaded. If records with higher keys are to be added to the end of the file, they are written to the last CA. When that CA becomes full, new CAs are formatted. Eventually, assuming it was allowed within the DEFINE, the VSAM file would start to take additional secondary extents to cater for new ‘high-key’ records being added to the file.

CA CI CI CI CI CI

CI CI CI CI CI

CI CI CI Ci CI

CI CI CI CI CI


13

DASD Utilization

In the Introduction, we suggested that VSAM’s inefficient DASD utilization can be as much of a problem to users as the performance issues highlighted above. Indeed, the two problems can often go hand-in-hand. For example, PART TWO of this Guide shows how increasing the freespace values for the file can reduce the CI/CA splitting described above, but can then lead to excessive DASD usage/wastage.

In addition, VSAM also wastes DASD space in the following situations:

• When the file contains records with Large Keys. VSAM can waste CIs in every CA when handling these.

• When inefficient blocksizes are chosen. The blocksize of a VSAM KSDS must be a multiple of 512 or 2048 bytes. This means that KSDSs are often created with blocksizes that are not the most efficient for the DASD device on which they reside.

• When a small Index CISIZE is used in conjunction with a small Data CISIZE. In the Index component, a CI must index all of the CIs within an individual CA in the Data component. If the Data CISIZE is small (i.e. lots of CIs in the CA) the Index CI referencing it can fill up, leaving the remaining CIs in the Data CA unused.

Later, in PART THREE of this Guide, we show how IAM’s efficient file structure, default blocking factors, and optional Data Compression and Space Release features can be used to utilize DASD more effectively.

Using Buffers

When the data in a VSAM file is to be processed, either by an online system (e.g. CICS) or by a batch job, the data records are moved into areas of virtual storage called ‘buffers’. Holding

records in these buffers eliminates the need for I/O operations (to disk) to access the data records in the file.

The number of buffers that can be used to hold INDEX and DATA records can be controlled manually by the user, either through the BUFSP parameter on the IDCAMS DEFINE, through the AMP parameter in JCL, or hard-coded within the actual accessing program itself.

Although it may be generally true that the more buffers made available, the better the performance of a VSAM file will be, this is not always the case. Indeed, it is important to take into account the type of access being done against the file—sequential or random. This often varies between CICS during the day and batch processing at night. It is not, therefore, sufficient to give all VSAM files maximum buffers; this would have a serious impact on the overall consumption of storage. More care has to be taken to ensure that the files that need the buffers get them, and the ones that do not, will not.


14

DFSMS VSAM System Managed Buffering (SMB) and the LSR Pooling functions under CICS, go some way to automating the buffer allocation process for VSAM, but these mechanisms often fail to provide optimum results, especially when there is a mix of sequential & random processing.

PART TWO of this Guide looks in more detail at VSAM Buffering; PART THREE shows how IAM can alleviate the problem of buffer tuning and improve on the overall performance of VSAM, even when manual tuning methods, or SMB or LSR tuning, have been applied to the VSAM files.

THAT COMPLETES OUR BRIEF OVERVIEW OF VSAM. IN PART TWO, WE MOVE ON TO HIGHLIGHT THE PROBLEMS INHERENT WITHIN THE VSAM FILE STRUCTURE AND EXPLAIN THE DIFFICULTIES FACING THOSE RESPONSIBLE FOR THE MANAGEMENT, CONTROL AND TUNING OF VSAM FILES WITHIN AN APPLICATION.


15

PART 2 – VSAM Problems This section describes the probelms in VSAM.

Introduction

PART ONE of this Guide gave a brief overview of VSAM and showed how the internal file structure is the root cause of the most common VSAM problems–namely Performance and DASD Utilization. Here, in PART TWO, we take a closer look at those problems and assess the implications they have on the Applications that use VSAM.

CI/CA Splitting

PART ONE described the concept and resulting inefficiency of VSAM’s CI/CA splitting mechanisms. This inefficiency can be described as:

• An overhead on CPU usage and I/Os, both to complete the split and to process the records that have been placed in a split CI/CA.

• An increase in the Response Times of online systems and Elapsed Times of batch jobs as a result of the delays during the split, and subsequent processing of a split file.

• A potential exposure to data integrity. VSAM clusters can become unusable if an insert is interrupted during a CI or CA split because records in the CI or CA being split may be lost if the split does not complete successfully.

One method used to counteract CI/CA splitting is to increase the freespace definitions in VSAM files that are to take a high number of inserts, with a view to decreasing the number of CI/CA splits.

For example, ‘FREESPACE(50 50)’ could be coded to request that 50% of each CI be left free and 50% of all CIs in every CA be left empty (see below, left). This would, however, only reduce the number of CI/CA splits that occur if the new records are being inserted evenly throughout the file. This is rarely the case! Often (see below, right), the insertion of new records is concentrated around a specific range of keys. Freespace cannot be graduated around the file,


16

so if the inserts are clustered, splitting still occurs around the area of insert activity, while other areas of freespace go completely unused.

An alternative for reducing the effect of CI/CA splitting is to reorganize the file on a regular basis. Reorganization is a process where IDCAMS reads all of the data records from the KSDS containing CI/CA splits and reloads them in proper key sequence, replenishing the freespace areas as it goes. Depending on the size of the KSDS, a Reorganization can take a considerable amount of time (i.e. tens of minutes) and while it is taking place, no other users or applications can access the file.

As we pointed out in the Introduction to PART ONE, VSAM files are often used in very “high- profile” systems, for which such prolonged unavailability is undesirable and, in extreme cases, can result in a loss of company business. The need to REORGANIZE therefore creates two major headaches for the personnel responsible for the performance and availability of VSAM in the systems concerned:

• REORG vs PERFORMANCE

On the one hand, they can keep the file available ‘non-stop’ for several days, at the expense of letting it split and suffer ever-decreasing performance. Alternatively, they can opt to REORGANIZE the file(s) each night, potentially causing a delay in the availability of data and, in addition, adding work to an overnight schedule that may already be stretched to the limit.

Example CA from a file with FREESPACE(50 50) coded.

Clustered inserts around a specific range of keys results in heavy splitting in that area of the file, while large area of freespace go unused elsewhere in the file.


17

• SELECTING FILES FOR REORG

The selection of files for REORG is a manual task with IDCAMS and is open to error. Some files that really need reorganization may, for example, get missed (and continue to perform badly), while others that don’t need reorganizing still get REORG’d (causing unnecessary downtime and adding to the total REORG time for the system).

The FDRREORG component of the FDR DASD Management System can be used to help with these problems. First, it automates the selection of files requiring reorganization (based on user criteria) to ensure that all the files that need REORGing are picked up, while those that do not can be bypassed. This ensures their continued availability and reduces the total REORG time.

Secondly, FDRREORG can then do the actual reorganization of the selected files much faster than IDCAMS. See PART 4 – FDRREORG for more information on the functions and benefits of FDRREORG.

PART THREE of this Concepts Guide shows how IAM can also be used to further reduce the effect of these problems by handling inserted records in a much more efficient manner, thus reducing the processing overhead when REORGs are run.

DASD Usage

As highlighted in PART ONE (and again earlier in this section), VSAM is not the most efficient user of DASD Space. Its freespace mechanism can waste large areas of space, unless records are inserted evenly throughout the file. In addition, VSAM has a limited ability for specifying the blocksize of files for optimum DASD usage.

DASD space may be cheaper ‘by the GB’ these days, but few companies can afford to unnecessarily waste it. And, of course, it is never just a case of the purchase price of the DASD. Consideration also has to be made on the increased hardware maintenance costs, the increase in backup times for the ‘over-large’ files and, perhaps more importantly, the increase in restore and disaster recovery time.

With an increasing demand for keeping more historical data online, the main files in some application systems can easily grow to several Gigabytes in size. When VSAM files reach this size, performance problems and poor DASD utilization becomes all the more apparent.

PART THREE of this Guide describes how IAM can be used to:

• Store data in significantly less space than required for VSAM. • Improve the performance on larger files.

Another ‘DASD Usage’ issue that is a common problem for VSAM is Over-allocation. It is difficult to calculate the space required for a VSAM file. Users often over-allocate, or use a standard


18

IDCAMS DEFINE job to create their VSAM files, without first altering the space allocation to match each individual file’s requirement.

This issue cannot be ignored. It is the aim of all DASD Managers and Storage Administrators to ensure that their DASD Space is being used efficiently and they carefully monitor the thresholds of their DASD volumes. However, because of a lack of suitable tools, many do not look at the actual usage of the allocated space within the files themselves. The wasted space inside VSAM files caused by over-allocation and the CI/CA freespace mechanism can often go unnoticed. It is almost impossible, for example, to find out how much of the CA freespace in a file has been used. Even if the DASD wastage problems are recognized and fully understood, the ‘cure’ is usually a reduction in the freespace definitions, at the expense of more CI/CA splits and the resulting performance degradations described earlier.

Buffer Usage

PART ONE introduced the concept of VSAM Buffering. In this section, we now take a closer look at the methods available to control buffer allocations of VSAM files and discuss their shortcomings.

NSR (Non-Shared Resources)

The original and default VSAM buffering mechanism is called NSR (Non-shared Resources). The default number of buffers provided by VSAM, unless otherwise specified, is the minimum required for VSAM to process the data set, and is woefully inadequate for most applications.

However, even with an adequate number of buffers provided, NSR buffering is ideally suited to predominately sequential processing. With NSR, two routes are available to decide how many buffers to allocate to a given VSAM file:

• Someone can look at the activity of the VSAM file and try to assess its buffer requirements. This can cause problems if the file is accessed by several programs, which may require different buffering (e.g. some doing random I/O, others doing sequential). It also assumes a regular, manual ‘check-up’ is made to ensure that the buffer requirements set previously are still valid, particularly if the use of the file has altered in some way.

• Alternatively, a ‘rule-of-thumb’ mechanism can be used, where buffering is estimated on a past history basis of another, similar file. This may lead to the new file getting insufficient buffers, or far more buffers than it requires. Neither result is satisfactory. With insufficient buffers, the file will not perform as well as it could. With too many buffers, it will be using resources (i.e. storage) that could be better utilized elsewhere.


19

A further complication arises when there are insufficient skills within a site to carry out the ‘manual’ buffer tuning described above. Most data centers now have hundreds, if not thousands of VSAM files, so manually tuning them becomes a full-time job!

LSR (Local Shared Resources)

VSAM offers a second buffering mechanism called LSR (Local Shared Resources). LSR provides a pool of buffers at the address space level that can be shared between multiple VSAM files and is ideally suited to random processing.

LSR processing requires that the application program build a buffer pool using the BLDVRP macro prior to opening the VSAM data sets that are to use the LSR buffer pool. The Batch LSR subsystem offers a mechanism via JCL to use LSR buffering for VSAM data sets by specifications in the JCL.

LSR buffering is also available under CICS, but it needs to be set up and administered by someone familiar with both CICS and VSAM. Under LSR, The VSAM buffering mode is determined at OPEN time and cannot be revised until the data set is closed and reopened. If the buffering mechanism is not selected wisely, the results can be very slow performance, with higher CPU times and excessive I/O is performed.

VSAM System Managed Buffering

VSAM System Managed Buffering, often referred to as SMB, provides some enhancements to VSAM buffering. SMB can select the type of buffering (NSR or LSR) and the number of buffers to use for the data and index components. When ACCBIAS=SYSTEM is set either in the Data Class or via JCL, SMB automatically selects a buffering mode, primarily based on settings in the VSAM ACB when the data set is opened. Additional parameters specified via JCL or the Data Class provide extra control.

Most of the “modes” of SMB buffering will, in effect, use VSAM NSR buffering, with an increased number of data and index buffers. The Direct Optimized (or DO) mode of SMB uses LSR buffering.

SMB limits the amount of storage it acquires for buffers so that it is only using a portion of the above the line region available. Batch jobs with lower region values may not have enough buffers to provide for optimal buffering, and may be forced into NSR buffering when LSR buffering would have performed much better.

So, in summary, even with SMB set to ACCBIAS=SYSTEM, users may still have to change JCL parameters, including the REGION on JOB / EXEC cards, or the AMP= parameter on DD cards, to obtain beneficial buffering on their VSAM files.


20

In PART THREE we show how IAM removes the requirement for manual buffer tuning and also how it copes much better (than VSAM) when there is a mix of random and sequential processing against the same file.

Record-Level File Sharing

Providing concurrent read/write access to a VSAM file across multiple jobs/transactions, either running in the same LPAR or across multiple LPARs can require quite extensive changes to the procedures and JCL of the programs that are doing the concurrent access.

In PART THREE we show how the IAM/RLS and IAM/PLEX features of IAM remove much of the set up and control requirements for implementing multiple concurrent access to VSAM files that have been converted to IAM.

Summary

Although some of the problems highlighted in the first two Parts of this Guide can, in part, be addressed by time-consuming manual effort, this is rarely feasible in today’s data centers where manpower is already stretched. Moreover, many of the ‘cures’ are simply a shifting of the problem from one area to another. Increasing freespace can, for example, reduce CI/CA splitting, but as we have shown, this ‘cure’ comes at the expense of an increase in DASD consumption and wastage.

It would appear to be a no-win situation for the personnel responsible for VSAM in a site. This is not the case! PART THREE of this Guide describes how IAM provides an alternative to VSAM, to address the problems we’ve highlighted above.


21

PART 3 – IAM This section describes the features available in IAM.

Introduction

PARTS ONE and TWO of this Guide highlighted the problems inherent in VSAM. They illustrated how the VSAM file structure can be stressed when used in the high-access, high-profile application systems of today. As such, the problems that were perhaps just minor headaches a few years ago have now become so acute that the manual ‘cures’ which have served well in the past are no longer sufficient. In many Data Centers there are now simply too many VSAM files to handle and too little time to spend manually tuning them.

Several products exist in the market, which try to alleviate this problem by removing some of the manual tuning effort. All of these products, however, ignore the fact that the fundamental problem of VSAM is the file structure. The only way to truly address the problems caused by VSAM is to move the data into an alternate, more efficient file structure.

IAM: The Power over VSAM

The Innovation Access Method (IAM) is a reliable, high-performance disk File Manager, designed as a transparent alternative to VSAM KSDS data sets. ESDS files are also supported as part of the base product (see ESDS Support), while the “IAM/AIX” cost option provides special support for VSAM Alternate Index (AIX) files (see Special Feature: Alternate Index (AIX) Support) and RRDS data sets.

IAM supports the concept of Record-Level Sharing (see Special Feature: Record-Level Sharing), allowing files to be shared for read and/or update between multiple CICS regions, TSO users and batch jobs running within a single z/OS LPAR. The additional cost IAM/PLEX option extends the capabilities to allow record- level sharing of IAM files across multiple LPARs in a z/OS Sysplex. Unlike the equivalent VSAM facilities, IAM’s record-level file sharing can be implemented with little or no changes required to the procedures and JCL of the accessing VSAM programs.

IAM often significantly outperforms VSAM under z/OS systems and it does not require any permanent modifications to the Operating System. It is designed to co-exist with VSAM without affecting the normal use of VSAM, and without interfering with VSAM Catalog Management.

The savings that IAM offers over VSAM can vary, but the figures below are an indication of the kind of savings that are typically achieved. Sometimes the savings are lower, but often they can


22

be higher, as was the case with the recent benchmark tests we did against VSAM SMB using standard KSDSs (Benchmarks (IAM vs VSAM with SMB)) and using a KSDS with two alternate indexes (Special Feature: Alternate Index (AIX) Support):

• 40-80% reduction in I/Os (EXCPs) • 20-70% reduction in CPU usage • 30-70% reduction in DASD space usage • 20-80% reduction in Batch Elapsed Times • 20-80% reduction in Online Response Times

User Experiences

“IAM reduced VSAM space requirements from 106,000 tracks to less than 53,000 tracks…”

“We replaced our existing VSAM compression package with IAM and saw a 50% saving in CPU time…”

“We implemented IAM for a key group of files rather than do a costly hardware upgrade…”

See PART 5 – Appendix for a list of vendor products that have successfully converted their VSAM files to use IAM.

In a moment, we will take a closer look at IAM to see how the above benefits are possible. First, we are going to see how IAM files can be used in place of VSAM without requiring any permanent changes to JCL, or to application programs, CICS, or even the Operating System!

The secret is the IAM/VSAM Interface: the VIF…


23

The VSAM Interface (VIF)

The VIF allows any program (e.g. Cobol, Assembler, PL/1, RPG) that uses keyed index access against a VSAM KSDS to load and then sequentially or randomly access IAM files without program changes. It is started at MVS IPL time, and dynamically installs itself in the SVC table. It then intercepts the SVC 26 (Locate), VSAM OPEN, VSAM CLOSE (TYPE=T), and SHOWCATs to determine if the request is for a VSAM file, or an IAM file:

If the request is for VSAM, the normal IBM routines are given control and the file processing continues as normal. If the file is IAM, the OPEN of its ACB is simulated to look like a VSAM OPEN. Then, when an online system or batch application program issues a record request against the file (e.g. GET, PUT, POINT, etc.) the request is processed by the VIF to look like the VSAM requests.

Creating an IAM File: IDCAMS Support

The VIF is also designed to intercept IDCAMS DEFINEs if they are ‘tagged’ as being DEFINEs for IAM files. This allows IAM files to be created using regular IDCAMS DEFINE jobs, despite having an entirely different file structure. (Note: The other methods for allocating VSAM files—JCL, SMS, etc.—can also be used to create IAM files. However, as in PART ONE, we assume that IDCAMS is the preferred method). The REPRO, DELETE, PRINT, and LISTCAT functions of IDCAMS are also supported for IAM files.

Because IAM files can be DEFINE’d with IDCAMS, this allows for a very easy migration from VSAM to IAM. Indeed, as VSAM files are usually DELETE/DEFINE’d on a regular basis during the Reorganization process, the IDCAMS DEFINE jobs are already available. It is a very simple task, therefore, to create IAM files with those jobs.

As you will see later, some of the parameters of the IDCAMS DEFINE are not relevant to an IAM file, but these parameters do NOT have to be removed and are ignored when the IAM file is

VSAM

IAMApplication

Program VIF

z/OS


24

created. Other parameters (e.g. FREESPACE) may be altered by IAM to take into account its more efficient handling of data. This is also explained later in this section.

Using IDCAMS, there are 2 methods (shown below) for ‘tagging’ a DEFINE to create an IAM file.

• Placing ‘$IAM’ in the OWNER field • Placing the characters ‘$IAM’ anywhere within the file name

Note: In the UK, the tag is £IAM

Either of the above methods can be used each time an IAM file is created. The ‘owner’ method is the most transparent because the actual filename does not change and, hence, the JCL of jobs that processes it run without requiring any alteration.

The ‘filename’ method can be chosen instead, if there is a particular need to identify files as IAM by their filename. IAM files do not have separate ‘.DATA’ and ‘.INDEX’ components to make them easily identifiable–the index portion of the IAM file is contained within the main file allocation itself. Regardless of the method chosen, the VIF intercepts the DEFINE and an IAM file (with a z/OS DSORG=PS) is created.

In the examples below, the result is a non-VSAM data set allocated on MVS001 with the name of ‘MY.TEST.DATASET’ or ‘MY.TEST.$IAM.DATASET’. This is an example of creating a single index IAM KSDS. See later for details on creating IAM ESDS and Alternate INDEX (AIX) files.

The ‘OWNER FIELD’Method The ‘FILENAME’ Method

//DEFVSAM EXEC PGM=IDCAMS //DEFVSAM EXEC PGM=IDCAMS //SYSPRINT DD SYSOUT=* //SYSPRINT DD SYSOUT=* //SYSIN DD * //SYSIN DD * DEFINE CLUSTER DEFINE CLUSTER (NAME(MY.TEST.DATASET) - (NAME(MY.TEST.$IAM.DATASET) - VOLUMES(MVS001) - VOLUMES(MVS001) - OWNER($IAM) - SPEED REUSE) - SPEED REUSE) - DATA(RECORDS(5000 500) - DATA(RECORDS(5000 500) - FREESPACE(10 20) - FREESPACE(10 20) - KEYS(10 0) - KEYS(10 0) - RECORDSIZE(256 256) - RECORDSIZE(256 256) - CISIZE(4096)) CISIZE(4096)) /* /*

IAM data sets can be defined as DFSMS extended format (EF) sequential data sets, allowing them to fully utilize “large” devices such as the 3390-9, 3390-27 and 3390-54. As an alternative, IAM files can also be defined as Large Format Sequential Data sets (introduced with z/OS 1.7). Unlike DFSMS EF, Large Format data sets do not have to be on SMS managed volumes and the 32-byte suffix to each data block is eliminated. Allocating an IAM file as either DFSMS EF or


25

Large Format, coupled with the placement of the file on “large” devices, raises the theoretical maximum size of an IAM file to almost 3TB of compressed data.

IAM Overrides

Some of the optional features of IAM are requested at file creation time. This is done by coding an IAMOVRID DD in the IDCAMS DEFINE JCL. The example below shows how to request Hardware Data Compression (see Data Compression) on all files being DEFINE’d in the particular step.

There is also an ‘ACCESS’ Override statement available to request further IAM features at file access time (e.g. Dynamic Tabling). These features are discussed later in this section. Let’s take a look now at the internal structure of an IAM file…

The IAM File Structure

PART ONE of this Guide described the internal structure of a VSAM file and illustrated its inefficiency when handling randomly inserted records. The IAM file structure, on the other hand, is designed to take inserted records more efficiently.

When allocated, an IAM file is divided into logical blocks of a size determined by the CISIZE parameter in the IDCAMS DEFINE, or via the BLKSIZE parameter coded on a CREATE override statement. These blocks are contained in the Prime Data Area of the IAM file. Unlike VSAM, IAM files are not limited to blocksizes of multiples of 512 or 2048 bytes. They can be created with an optimum blocksize for the device on which they are being allocated. IAM files typically have blocksizes of 1/4 or 1/2 track.

IAM Data Blocks are usually much larger than VSAM CIs and are not contained in anything equivalent to a CA. However, when loaded, they are used in the same way as a CI. Records are written into each block in key sequence and, if CI freespace was coded on the DEFINE, an

//DEFVSAM EXEC PGM=IDCAMS //SYSPRINT DD SYSOUT=* //SYSIN DD * DEFINE CLUSTER (NAME(MY.TEST.$IAM.DATASET) - VOL(MVS001) - KEYS(9 0) - RECORDSIZE(1024 1024) - CYLINDERS(500 200) /* //IAMOVRID DD * CREATE DD=&ALLDD.DATACOMPRESS=YES /*


26

equivalent percentage of freespace is reserved within each block. This freespace is known as Integrated Overflow, indicated by the shaded areas below:

As new records are inserted into the IAM file after the initial load, they are placed in the appropriate Data Block, using the Integrated Overflow available. This, however, is as far as the similarity with VSAM goes. If the Integrated Overflow has been exhausted in a block, IAM does not do a ‘split’. Instead, the new record is placed into an Extended Overflow block at the end of the file:

IAM’s Extended Overflow Blocks replace the function of VSAM CA Freespace. Unlike CA Freespace, however, Overflow Blocks are not tied to specific Prime Blocks. They can be used for any inserts that cannot fit into a Prime Block. They are also structured to cater for variable length records, giving optimum efficiency when the Maximum Record Length for the data records in the file is significantly larger than the defined Average Record Length.

All of this means that less DASD space is required to handle random insert activity when compared to VSAM. When a VSAM file is converted to IAM format, its size can be significantly reduced. Typically, the size of a VSAM file can be reduced by 20-40% simply by converting it to IAM format.

Data Block

Data Block

Data Block

Overflow Block


27

Inserts to the end of an IAM file (i.e. new high keys) go into Extended PE blocks. They are similar to Extended Overflow Blocks.

An IAM file is ‘self-describing’. Its control information (e.g. Blocksize, RECFM, LRECL) is stored at the front of the file, followed by the Prime Data Area which contains data loaded by the initial REPRO into the file. The Data Blocks in the Prime Data Area may have Integrated Overflow freespace reserved in them to cater for inserted records.

After the Prime Data Area comes the Index to the Prime Area. This is an efficient index that is held in storage when the file is opened to reduce DASD I/O when processing the file.

Finally, if the Integrated Overflow in a Prime Data Block has been exhausted, or if records have been added to the end of the file, you find Extended Overflow and Extended PE blocks at the end of the IAM file. Secondary extents are taken by IAM (if coded) when additional DASD space is required for these overflow blocks.

The IAM “PRO” File Structure

The Prime Related Overflow structure (PRO), introduced with IAM 8.1, is an option for IAM files that are subjected to an extremely high volume of records being inserted, especially if those inserts are clustered around a certain key range.

IAM data sets that use the optional PRO structure can require significantly reduced amounts of virtual storage for jobs that process the file, and can also improve performance, both at insert time, and later when the file is further processed.

The use of the PRO structure may also have a knock-on effect of reducing the frequency with which a “heavy insert” IAM file would need reorganizing (see File Reorganization) on the next page.

Control Information

Prime Data Area

Index

Extended Overflow

Extended PE


28

File Reorganization

Despite IAM’s more efficient method of handling inserted records, IAM files still require a reorganization from time to time to move records from the Overflow Blocks (and Integrated Overflow) into the Prime Data Blocks. An IAMW22 warning message appears in the IAMINFO report for a file (see IAMINFO–Execution Time Statistics) recommending that a reorganization be carried out.

This reorganization can be done using the same IDCAMS REPRO jobs that were used for the equivalent VSAM files. The basis for ‘REORG selection’, however, is the amount of Overflow usage, rather than the number of CI/CA splits, as under VSAM.

Because of the nature of IAM, IDCAMS REORGs of IAM files usually run faster than a REORG of the equivalent VSAM file. A feature in IAM called ‘Backup Compressed’ allows the reorganization of compressed IAM files to take place without needing to de-compress and re-compress the data on backup and reload. This saves a considerable amount of CPU and elapsed time on reorganization—particularly on large IAM files.

If REORGs are still taking too long, however, you may wish to apply more intelligence to your IAM (and VSAM) reorganizations by using the FDRREORG component of the FDR DASD Management System. As described in PART FOUR, FDRREORG allows for the selection of VSAM/IAM files for reorganization based on user specified criteria (e.g. “percentage of CI and/or CA splits” for VSAM, or “number of overflow records” for IAM).

This intelligent selection within FDRREORG ensures that only the files that really need REORGing are reorganized.

Transporting IAM Files

IAM files can be transported from one location to another using a standard IDCAMS REPRO to create a sequential copy of the file on disk (perhaps to be e-mailed or FTP’d) or to tape (if it is to be physically transported from one location to another).

Once received at the alternate location, the sequential copy of the data can then be loaded to a new IAM file, again using a standard IDCAMS REPRO.

Data Compression

In addition to the space savings achieved with IAM’s efficient file structure, the data within an IAM file can also be compressed to provide further space reductions. As well as saving space, Data Compression also reduces I/Os to the file, which has a knock-on effect on CPU usage


29

because it costs CPU to do I/Os. The DATACOMPRESS override (see IAM Overrides) is used to request data compression for the file, as explained below.

Software Compression

Requesting the Software Compression feature of IAM can reduce the DASD requirement for an IAM file by an additional 20-50%. Although CPU is expended doing the de-compression and re-compression of the data, IAM’s efficient compression routines (and the reduction in CPU from the reduced I/Os) usually keeps the overall CPU usage below that of the original uncompressed VSAM file.

Software Compression is requested with the DATACOMPRESS=SW override when creating (or re-loading) the IAM file.

Hardware Compression

IAM also supports System z Hardware Compression function, via the DATACOMPRESS=HW override. IAM dynamically creates a hardware compression dictionary as the file is being loaded. It temporarily holds, in a data space, up to the first eight megabytes of data being loaded. Once the target quantity of data is present, it then invokes its internal dictionary build process using the initial data records from the data space.

After the dictionary is built, the records in the data space are compressed and physically written out to the IAM file. Then, the remainder of the data is compressed as it is being passed to IAM and physically written to the file. The hardware compression dictionary is stored within the IAM data set. As an alternative, you may optionally override the dictionary build process and provide your own dictionary, controlled by the DICT=override.

WARNING: THE USE OF HARDWARE COMPRESSION ON AN IAM FILE IS NOT RECOMMENDED ON OLDER PROCESSORS, AS THE ADDITIONAL CPU TIME CAN BE SUBSTANTIAL.

Space Release

Unused space within an IAM file can be released after the file has been loaded or reloaded. This cures the problem of ‘over-allocated’ files that we discussed in PART TWO. Because IAM files generally take 30-70% less space than VSAM, if the original space allocations are left unchanged in the DEFINEs during the conversion, the unused space can be released.

IAM first checks to ensure that a secondary allocation has been defined for the file before performing the Space Release (to prevent immediate Sx37 abends) and also leaves a percentage of freespace after the Release to avoid immediate Secondary Space allocation.


30

Support for PAV

IAM provides full support for Parallel Access Volume (PAV)–a feature of some DASD devices which provides for concurrent physical I/O activity against the same physical z/OS DASD volume. Use of this capability can provide substantial improvements in response times for online systems or batch jobs that have heavy concurrent I/O activity.

Additional Performance Features

So far, we have concentrated on the various features of IAM that reduce DASD Space requirements for a file. We have also shown how IAM’s file structure can reduce I/Os and CPU by eliminating CI/CA splits, and how the “Data Compression” and “PAV support” features can further reduce I/Os to assist with the performance of a file. In addition to all of the above, IAM also provides several other key performance features that offer a reduction in I/Os and CPU, thus further improving Elapsed Time or Response Time performance.

In-Core Index

When an IAM file is OPEN’d, IAM reads the entire INDEX of the file into Virtual Storage and all subsequent references to the index are made to the ‘in storage’ copy. This essentially eliminates any I/O to the INDEX. VSAM files that have a high ratio of activity to the index component (compared to the data component) usually yield very significant savings when converted to IAM.

Real Time Tuning

Through its ‘Real Time Tuning’ mechanism, IAM also dynamically tune the allocation and use of DATA buffers to match the demands of the processing program. IAM continually monitors I/O activity as a file is processed and applies the artificial intelligence concept of ‘learning by experience’ to the file processing.

This process, of course, differs depending on whether random or sequential processing is being done. IAM determines, and then uses, the number of Data buffers that result in the best level of performance for the file. It then continually monitors buffer usage and determines from its experience if any additional buffers would decrease real I/O. If so, it acquires them. The optional “TURBO” mode results in a much more aggressive acquisition of buffers during periods of heavy I/O activity. This dynamic tuning of Data buffers eliminates the need to manually calculate, code, and monitor buffer allocation. The files that need more buffers get them and those that do not, will not.


31

For multi-volume files, IAM also initiates multiple-concurrent I/Os to further improve I/O performance (including file Reorganizations on multi-volume IAM files with FDRREORG).

Dynamic Tabling

IAM can optionally extend the buffering process by creating a ‘table’ in virtual storage of the records that are read from a file. On random reads, IAM checks to see if the requested record is already in its table and if it is, passes it back to the application, eliminating the I/O to the disk. If the record is not currently in the table, IAM reads it from the file, passes it to the application and tables it for subsequent retrievals. If the record is updated by the application, IAM changes the record in the table and on disk.

Files that gain the most benefit from Dynamic Tabling are those where the same records are read and re-read in a short space of time, and where few are being updated. Small files with high random READ activity effectively become in-core tables.

At OPEN time, a table is requested in one of two ways–with either a DYNCORE or DYNDS override. With DYNCORE, the storage for the table is obtained from within the accessing job or CICS region. With DYNDS, the table is allocated in a z/OS data space, allowing much larger amounts of virtual storage to be used.

IAM’s Dynamic tabling is very flexible, because some files may require a table for certain processing (e.g. during the ‘Online’ day) and not at other times (e.g. Overnight batch).

The Dynamic Table can therefore be requested only when needed.

In the example below, the IAM file pointed to in JCL by the FILE3 DD card is allocated with a Dynamic Table of 10MB, which is placed in a data space. The file’s record size determines the maximum number of records that can be tabled at any one time. Note that we have also specified the TURBO mode.

See Benchmarks (IAM vs VSAM with SMB) for IAM vs VSAM benchmarks.

IAM under CICS

IAM files can be used under CICS systems without requiring any change to IAM or to CICS. The FCT entry for the file can go unchanged. If the file is defined to an LSR Buffer pool, the definition can remain unchanged. Although IAM does not use the LSR buffer pool, it can take advantage of other LSR related CICS services which improve CPU usage. The only potential change to CICS

//IAMOVRID DD * ACCESS DD=FILE3,DYNDS=10,TURBO=YES /*


32

would be a reduction in the size of LSR buffer pools. As files get converted to IAM, less LSR buffers are required for the remaining VSAM files.

IAM supports the BWO (Backup While Open) service, which is primarily used by CICS and CICS/VR to allow for backing up open files, and then for subsequent forward recovery after the data set has been restored. BWO data includes three flags, indicating the BWO state of the file, and the 8-byte date and time stamp field, which is used as a file recovery starting point. IAM maintains this BWO data, which can be printed on an IAMPRINT LISTCAT report.

IAMINFO–Execution Time Statistics

With the potential DASD, CPU, I/O and Elapsed/Response Time savings that can be obtained from IAM, it is useful to be able to monitor and report on these savings. This can, in part, be achieved by requesting the optional ‘IAM Execution Time Statistics’ report, produced by coding an IAMINFO DD in the accessing JCL.

Once coded in a JCL step, the IAMINFO DD causes this statistics report to be produced for each CLOSE of an IAM file in that step. The report includes information such as:

• File Attributes (BLKSIZE, LRECL, RECFM) • Buffer usage (Number Used, Size, Storage Required) • File Activity (Reads, Writes, Deletes, Inserts) • Extended Block usage (Overflow Blocks, Extended PE blocks)

The IAMINFO report also indicates:

• The size of the IAM file, so that during a VSAM-to-IAM conversion, a clear indication is given of the DASD savings made.

• The number of I/Os that were saved due to requests being satisfied by the buffers or Dynamic Table. The report also indicates if IAM was unable to obtain additional buffers for a file due to upper buffer limits or storage constraints.

The IAMINFO report can also include a recommendation when the IAM file needs a reorganization (i.e. because it has a large number of records in overflow, or it has taken a large number of secondary extents on a single volume).

More detailed reports can also be produced by writing the IAMINFO data as SMF records, which can then be processed by additional reporting programs supplied with IAM (see Identifying Candidates for IAM Conversion). A report similar to an IAMINFO can also be produced in real time under CICS or TSO. The IAMXMON transaction can supply information on IAM files currently open to a CICS system.


33

Special Feature: Record-Level Sharing

In today’s 24 x 7 environments it is no longer viable to restrict a file to the rigid protocol of “one-at-a- time access”. These days, in order to fit increasing amounts of work into decreasing windows of time, data centers need to be able to provide concurrent access to certain files, whenever they want and from wherever it is necessary.

Here are two typical comments that we often hear from customers:

“We need to run batch work during the day against our CICS VSAM files. We want to be able to do this without having to take down CICS or close the files

to CICS…”

“We want to be able to run our batch work whenever it is required, and we want to run it against the live versions of our CICS VSAM files (i.e. not copies) so that our end-users receive batch reports containing the very latest data.”

Clearly, the issues of sharing VSAM files between CICS and one or more batch jobs (or other VSAM applications) is becoming increasingly important to many companies. For this reason, IAM is continually being developed to address these needs, and its Record Level Sharing capabilities are specifically designed to allow users to concurrently access the VSAM files that they have converted to IAM.

IAM Record-Level Sharing–Summary

The two sections below describe the facilities available in IAM for Record-Level Sharing.

• Record-Level Sharing for a single system (IAM/RLS) was introduced in IAM Version 8.0 and allows an IAM file to be concurrently accessed by two or more applications running in the same LPAR.

• Record-Level Sharing for a SYSPLEX (IAM/PLEX) was introduced as a separate cost-option in IAM Version 9.0 and allows an IAM file to be concurrently accessed by two or more applications running in either the same LPAR or different LPARs.

IAM Record-Level Sharing–Single System (IAM/RLS)

IAM/RLS allows batch jobs, TSO users, and CICS regions that are running on the same LPAR to safely access and update an IAM file at the same time. Eligibility for concurrent processing under IAM/RLS is determined during OPEN processing, on a file-by-file basis.


34

As described earlier, and also illustrated below on the left, traditional (i.e. Non-RLS) access to an IAM file is handled through the VIF. Although multiple VSAM applications are able to concurrently access an IAM, this can lead to unpredictable results and is not recommended.

As shown on the right, the IAM/RLS address space sits between the VIF and the IAM file, and it

handles the concurrent access to that file from multiple VSAM Applications running on the same LPAR. These applications can be CICS transactions, TSO users or batch jobs, and they can be accessing the IAM for read and/or write processing.

IAM can automatically decide during OPEN processing which IAM data sets are to be handled by IAM/RLS. This can be determined based on:

• The share options that a data set was defined with • An RLSID operand specified when a data set is defined or loaded • By an entry in a special IAM/RLS data set name table.

Depending on the RLS selection criteria implemented at an installation, there are generally little or no JCL changes required to use IAM/RLS. For CICS, the installation and activation of certain

VSAM Application

VIF

IAM File

LPAR:001

Traditional (Non-RLS) IAM Access

VSAM Application

#1

IAM/RLS

IAM File

LPAR:001

VIF

VSAM Application

#2

IAM/RLS – Concurrent Access on the Same LPAR


35

exits is required. For batch jobs that perform a large volume of updates to recoverable files being processed by IAM/RLS, the implementation of the IAM Batch Syncpoint (Record Locking) and appropriate restart capabilities are highly recommended.

IAM/RLS does not require the use of a coupling facility or SYSPLEX services.

IAM/RLS includes an ISPF interface to monitor activity within the IAM/RLS address space, and issue some of the operator commands.

Record Locking

Locking during update processing is performed at the record level, utilizing the IAM/RLS record locking facility. For recoverable files, record locks are held until the job or transaction terminates, or the job or transaction issues a SYNCPOINT. Potential deadlocks within the resources managed by IAM/RLS are detected, and the associated request is failed to prevent a deadly embrace.

Journaling, Recovery & Backout

Journaling and recovery capabilities (see Journaling & Recovery below) are provided for data sets processed under IAM/RLS and optionally for IAM data sets that are not processed under IAM/RLS.

IAM/RLS can dynamically backout updates made to files when the batch job step that performed those updates abends. A callable batch syncpoint process is provided, to prevent batch programs from locking out access to large portions of recoverable files when mass updates are being performed.

Multiple IAM/RLS Address Spaces

As illustrated below, you can have multiple IAM/RLS address spaces on a single LPAR or z/OS system image. This is useful if you need to isolate different applications for testing, security or accounting purposes. Each IAM/RLS address space is assigned a four-character identifier


36

(referred to as the RLSID), and you can select which RLS address space is used for particular jobs or CICS regions.

IAM Record-Level Sharing–On a SYSPLEX (IAM/PLEX)

IAM/PLEX, a separate cost-option to the base IAM license, utilizes the z/OS Sysplex XCF services and the z/OS System Logger facilities to enable record-level sharing of IAM files across multiple LPAR’s that are part of a z/OS sysplex. VSAM applications that share these IAM files through IAM/PLEX can be any mix of CICS transactions, batch programs, or TSO users.

IAM/PLEX builds on the IAM/RLS framework described above, and provides users with record- level sharing capabilities without the implementation complexity and application changes that are typically required for similar file-sharing facilities on VSAM files.

LPAR:001

VIF

IAM/RLS #1 IAM/RLS #2

VSAM Application

#1

VSAM Application

#2

IAM Files #1 IAM Files #2

IAM/RLS – Multiple IAM/RLS Address Spaces


37

As shown in the diagram below:

• VSAM application #1, which is running on LPAR:001 is accessing an IAM file through the VIF and underlying IAM/PLEX address space, which is also running on LPAR:001.

• At the same time, VSAM application #2, which is running on LPAR:002, is also accessing the same IAM file.

• The IAM/PLEX address space running on LPAR:002 is communicating (though XCF) with the IAM/PLEX address space on LPAR:001.

Journaling & Recovery

IAM provides the facility to write BEFORE and/or AFTER journal records, which can be employed in either a “forward” or “backout” recovery of a file:

• A “backout recovery” is performed in the event of a failure/abend of a single (updating) job. At the point of failure, the “Before” images of the updated data records are written back to the IAM file, working backwards in time, thus removing all the updates.

VSAM Application #1

LPAR:001

VIF

IAM/PLEX

IAM File

VSAM Application #2

LPAR:002

VIF

IAM/PLEXXCF

IAM/PLEX – Concurrent Access Across Multiple LPARs


38

• A “forward recovery” is used if an older copy of an IAM file has to be restored, perhaps after encountering a media failure, or data corruption, or maybe even in the event of Disaster Recovery. Once the older copy of the file is back on disk, the “After” images can then be re-applied to the file, working forward in time, thus recreating the updates that had taken place.

These procedures, which have been available for some time in IAM, have been enhanced to accommodate the changes introduced by the IAM/RLS and IAM/PLEX.

IAM also features a “Dynamic Job Backout” (DJB) facility, which provides a controlled, automated and fully dynamic “backout recovery” of IAM/RLS and IAM/PLEX files. If an updating job step abends, all uncommitted updates (for that job) are automatically removed from all IAM/RLS or IAM/PLEX files being updated by that job. This negates the need to run separate, manual, batch-driven recoveries for each affected file.

Special Feature: Alternate Index (AIX) Support

As mentioned earlier, users of the “IAM/AIX” can also define one or more Alternate Indexes (AIXs) for the KSDS, RRDS and ESDS files that they have converted to IAM. The procedure for creating an IAM KSDS with an Alternate Index is the same as for VSAM:

• Define the base IAM file with the DEFINE CLUSTER command (using the $IAM identifier) • Define the AIX with DEFINE ALTERNATEINDEX • Build the AIX with BLDINDEX • Define the PATH with DEFINE PATH

IAM files with AIXs provide the same sort of savings over VSAM as with regular IAM files. In some recent tests comparing IAM against VSAM SMB, we did some benchmarking on alternate index processing. Two alternate indexes were used, one with unique keys, and one with non-unique keys.

Here is a summary of the results of two of the AIX benchmark tests, the full details of which are available on request.


39

TEST 1 – RANDOM PROCESSING

This first test performed 256,000 random read and 160,000 update requests through one of the alternate index data sets. Half of the updates required an update to the other alternate index.

The processing also included 80,000 inserts, 16,000 deletes, 16,001 points (start browses) with 80,000 sequential reads (5 records read per start browse).

In comparing the results of IAM AIX processing versus VSAM we found that:

• IAM used 75% to 85% less CPU time • IAM ran in 84% to 96% less elapsed time • IAM used 92% to 99% less EXCPs

IAM’s Real Time Buffer Tuning resulted in a very significant performance advantage over VSAM SMB, primarily because SMB does not use any LSR buffering for VSAM clusters with alternate indexes.

TEST 2 – SEQUENTIAL READ (With Non-Unique Key Alternate Index)

This test was performed after TEST 1, and it read the entire base cluster through the non-unique key alternate index, in a total of 2,064,000 I/O requests.

This is a case where the ideal buffering situation is for the alternate index to use a buffering technique optimized for sequential buffering, and for the base cluster to be optimized for random buffering. This mix of buffering techniques is automatically handled by IAM, but it cannot be achieved with VSAM, as both the AIX and base cluster MUST use the same buffering mechanism.

The IAM advantages are, once again, quite clear from the results:

• IAM used 80% to 86% less CPU time • IAM ran in 88% to 97% less elapsed time • IAM used 93% to 99% less EXCPs


40

ESDS Support

ESDS files are also supported by IAM. Conversion of a VSAM ESDS into IAM is achieved in the same way as a KSDS-to-IAM conversion–using the $IAM identifier within the IDCAMS NONINDEXED define job.

Two examples of defining IAM ESDS files are shown here:

The ‘OWNER FIELD’Method The ‘FILENAME’ Method

//DEFVSAM EXEC PGM=IDCAMS //DEFVSAM EXEC PGM=IDCAMS //SYSPRINT DD SYSOUT=* //SYSPRINT DD SYSOUT=* //SYSIN DD * //SYSIN DD * DEFINE CLUSTER DEFINE CLUSTER (NAME(AN.ESDS.DATASET) - (NAME(A.$IAM.ESDS.DATASET) - VOLUMES(MVS001) - VOLUMES(MVS001) - NONINDEXED - NONINDEXED - OWNER($IAM) - TRK(45 10) - TRK(45 10) - RECORDSIZE(256 256) - RECORDSIZE(256 256) - SHAREOPTIONS(2,3)) SHAREOPTIONS(2,3)) /* /*

IAM provides the same advantages for ESDS files as highlighted previously with IAM KSDS and IAM/AIX data sets, namely; reduced DASD space utilization, reduced I/O and reduced batch elapsed times and online response times. These savings are achieved with the same facilities (Data Compression, advanced file structure, Real Time Buffer Tuning Management, etc.). IAM ESDS files can also have associated alternate index (AIX) data sets.

The internal structure of an IAM ESDS is similar to that previously described for IAM KSDSs. It also has the ability to provide Overflow space within the file. This space is required if the data in the file is compressed. There may be occasions where records increase in size due to a different compression after the update.

Because of this utilization of overflow within the IAM ESDS, the file from time to time requires an IDCAMS Reorganization, similar to that for KSDSs. The process of doing this Reorganization is the same as for KSDS–either a REPRO out to a sequential file and then back into the original file, or a simple REPRO out to a new file. Alternatively, as for VSAM KSDSs and IAM KSDSs, an IAM ESDS can be reorganized with FDRREORG, as described in PART FOUR.

IAM ESDS data sets can exceed 4 gigabytes as long as the application does not have a dependency on the RBAs (Relative Byte Addresses) of each record being the identical value as VSAM’s. This capability is provided by the PSEUDORBA keyword on an IAM CREATE Override when the file is defined or loaded. This indicates to IAM that it can generate a 4-byte RBA that


41

is different than the one VSAM would use for the same record. In effect, IAM returns a 4 byte relative record number.

Note: For applications that require a true 4-byte VSAM RBA value, such as the SAP product, the size limitation is still 4 gigabytes of user data.

IAM also supports 8-byte RBA values, also known as Extended Addressability, which was introduced with DFSMS 1.5. This allows ESDS files to exceed 4 gigabytes of user data while using a VSAM compatible RBA value. The IAM support for Extended Addressability does not require DFSMS 1.5, nor does it require the data set to be SMS managed. This support is triggered by specifying the XESDS keyword on the IAM CREATE Override.

Identifying Candidates for IAM Conversion

You may have some VSAM files which, although eligible for conversion to IAM, may only be very small or have a very low activity against them. These files would not yield any major savings when converted to IAM. When looking at implementing IAM, it is usually preferable, therefore, to target the eligible files that are the biggest and/or have the highest activity against them, before looking at the smaller and/or lower activity files.

It is not uncommon for just a few files in an application to be responsible for a very high percentage (e.g. 80-90%) of the DASD Usage and I/O Activity. For these reasons, a facility is provided to help identify the files which are both eligible for conversion to IAM, and which offers the greatest savings.

IAMSMFVS

Information about VSAM file size and activity is already being recorded via the standard SMF recording function of z/OS (record type 30 subtype 4, and record type 64). IAM utilizes this information and provides a comprehensive reporting utility, IAMSMFVS, to produce a list of the BIGGEST and MOST ACCESSED eligible files. IAM can also supply information on which jobs/steps use those files. This information is invaluable when assessing the conversion implications. IAMSMFVS can be run against the current SMF file or a cumulative history file contained on disk or tape. Here is a sample of the EXCP and SIZE reports that it produces:


42

Sample IAMSMFVS “EXCP” Report

VSAM EXCP REPORT

%CLUSTER TOTAL SPLITS ALLOC DATA SET NAME EXCPS EXCPS RECORDS READS INSERTS UPDATES DELETES CI CA TRKS ---- --- ---- ----- ----- ------- ----- ------- ------- ------- -- -- ---- SYSPCICS.CICSIDP1.DFHGCD 100 1727 SYSPCICS.CICSIDP1.DFHGCD.DATA 90 1552 655 5116 1968 394 1963 27 0 15 SYSPCICS.CICSIDP1.DFHGCD.INDEX 10 175 1 0 0 66 0 0 0 1 SYSPSMF.CPUB.MAN1 100 900 SYSPSMF.CPUB.MAN1.DATA 100 900 0 72546 0 0 0 0 0 90

SYSPCICS.CICSIDP1.DFHLCD 100 580 SYSPCICS.CICSIDP1.DFHLCD.DATA 98 566 260 2008 0 81 0 0 0 4 SYSPCICS.CICSIDP1.DFHLCD.INDEX 2 14 1 0 0 0 0 0 1 AJM.UPSTREAM.CATALOG 100 373 AJM.UPSTREAM.CATALOG.DATA 74 277 7017 13517 71 0 27 0 0 90 AJM.UPSTREAM.CATALOG.INDEX 26 96 7 0 0 0 0 0 0 1 SYSPRMM.MASTER 100 360 SYSPRMM.MASTER.DATA 96 344 4015 6092 0 0 0 0 0 285 SYSPRMM.MASTER.INDEX 4 16 9 0 0 0 0 0 0 1 IAM63.VSAMTEST.DATA 100 243 IAM63.VSAMTEST.DATA.DATA 67 162 1172 81 0 81 0 0 0 45 IAM63.VSAMTEST.DATA.INDEX 33 81 1 0 0 0 0 0 0 1

Sample IAMSMFVS “SIZE” Report

VSAM SIZE REPORT

ALLOC TOTAL USE AVG MAX KEY DATA SET NAME TRKS EXCPS COUNT EXTENTS LRECL LRECL LEN RKP CISIZE ---- --- ---- ----- ----- ----- ------- ----- ----- --- --- ------

AJM.UPSTREAM.FILEDATA 2705 0 2 AJM.UPSTREAM.FILEDATA.DATA 2700 0 2 4 840 6165 21 0 22528 AJM.UPSTREAM.FILEDATA.INDEX 5 0 2 3 505 21 0 512

SYSPSMF.CPUB.MAN1 900 900 5 SYSPSMF.CPUB.MAN1.DATA 900 900 5 1 22518 32767 0 0 22528

SID.UPSTREAM.FILEDATA 345 0 2 SID.UPSTREAM.FILEDATA.DATA 330 0 2 3 840 6165 21 0 22528 SID.UPSTREAM.FILEDATA.INDEX 15 0 2 1 4089 21 0 4096

SYSPRMM.MASTER 286 360 2 SYSPRMM.MASTER.DATA 285 344 2 1 512 9216 56 0 10240 SYSPRMM.MASTER.INDEX 1 16 2 1 2041 56 0 2048

USTEST.UPSTREAM.FILEDATA 151 4 3 USTEST.UPSTREAM.FILEDATA.DATA 150 1 3 1 840 6165 21 0 22528 USTEST.UPSTREAM.FILEDATA.INDEX 1 3 3 1 4089 21 0 4096

SYSPSMF.CPUB.MAN2 150 600 2 SYSPSMF.CPUB.MAN2.DATA 150 600 2 1 22518 32767 0 0 22528

AJM.UPSTREAM.CATALOG 91 373 2 AJM.UPSTREAM.CATALOG.DATA 90 277 2 2 50 500 18 0 22528 AJM.UPSTREAM.CATALOG.INDEX 1 96 2 1 2041 18 0 2048

IAM63.VSAMTEST.DATA 46 243 81 IAM63.VSAMTEST.DATA.DATA 45 162 81 1 255 300 8 4 22528 IAM63.VSAMTEST.DATA.INDEX 1 81 81 1 505 8 4 512


43

SMS Support

IAM files can be placed on DFSMS controlled volumes and are treated as non-VSAM data sets. As such, they are cataloged as non-VSAM and have an NVR in the VVDS. From then on, they are treated in the same way as any other VSAM or Non-VSAM SMS file, as follows:

• When allocated, they go through the ACS routines and SMS Volume Selection • They are managed as per the criteria of the Management Class to which they have been

assigned • They can be backed up, scratched or migrated by the SMS System’s Data Manager (e.g.

Compuware’s ABR or IBM’s DFHSM)

An additional benefit under SMS Systems is provided via the DATA CLASS facility of SMS. If a VSAM file is created with a DATA CLASS name containing $IAM, the DEFINE is intercepted by the VIF in the usual way and an IAM file is created. SMS Data Classes can make it simpler to standardize on file attributes such as Blocking Factor, which can be used to improve performance. For this reason, the Data Class method of creating IAM files may be preferable to placing ‘$IAM’ in the data set name or Owner field as previously described.

IAM Reliability

As mentioned in the Introduction to this section of the Guide, IAM is a very robust file manager.

It has been designed with the aim of preserving file integrity in the event of a system or application failure–even during insert processing.

• IAM relieves the integrity problems caused by VSAM’s CI/CA split concept. VSAM clusters can become unusable if an insert is interrupted during a split. Records in the CI or CA being split may be lost if the split does not complete successfully. The structural integrity of an IAM file is always assured. Barring physical damage to the disk, IAM files are logically indestructible. IAM’s Overflow concept allows records to be inserted without using splits. Records are added in a single disk write, without the need to update the index.

• IAM provides a recovery program. The ‘IAMRECVR’ program can be used to read data in an IAM file, even if portions of that file are physically damaged and/or the VIF is inactive. IAMRECVR attempts to recover as much data as possible, copying the undamaged portion of the file to either a sequential file, a new IAM file, or a new VSAM file.

IAM Installation

Despite the huge benefits that IAM provides over VSAM, it is a very easy product to install and get running.


44

The IAM program modules can be shipped via Electronic Installation.

Once the Installation Control Library (ICL) has been created from one of the above sources, a sample ‘INSTALL’ job is used to load the IAM modules into a z/OS APF authorized library.

The VIF is then started using the sample ‘VIFSTART’ job from the ICL.

These steps can easily be completed in less than an hour and, once done, IAM files can immediately be created by running IDCAMS DEFINEs with $IAM either in the filename or owner field.

IAM ISPF Panels

IAM also comes with a full set of ISPF panels to assist with the running of utility functions against IAM and VSAM files (e.g. Allocation, Delete, Copy, Rename). IAM’s report programs (such as IAMINFO described earlier) can also be invoked from the panels. Tutorials are also included.

The key features included in the IAM ISPF panels are:

• Definition of IAM data sets with full Override support. • Definition of IAM Alternate Indexes and Paths. • Definition of VSAM clusters. • Definition of VSAM Alternate Indexes and Paths. • Definition using an IAM or VSAM model data set. • Multi-Volume data set support. • SMS support. • Building IAM or VSAM Alternate Indexes. • Deletion of Data sets, Clusters, Paths and Alternate Indexes. • Renaming of Data sets, Clusters, Paths and Alternate Indexes. • Copy/Move support of IAM data sets, VSAM ESDSs and VSAM KSDSs. • IAM or VSAM data sets can be copied into or from sequential data sets. • Reporting on IAM files and VSAM clusters (data set information). • Interactive execution of selected IAM utility functions.

Users that are familiar with ISPF and that have a knowledge of IAM or VSAM should find the IAM ISPF panels easy-to-use. To illustrate some of the functions of the IAM ISPF panels, a demonstration follows below.


45

SUPERC Support

SuperC utility functions under ISPF can be used on IAM data sets. These are the file comparison and searching functions (SuperC, SuperCE, Search-For, and Search-ForE).

Defining an IAM File with the ISPF Panels

The following steps are required to define an IAM data set through the IAM ISPF Panels:

• Type an I in the Option field of the IAM primary menu • Fill in the data set name field • Type a C for the data set type for an IAM ESDS or KSDS type of file • Optionally, fill in the model data set name field of an IAM or VSAM KSDS or ESDS • Then press the ENTER key

An example of a completed panel is shown below, where we are creating an IAM file called “myiam.cluster”:

--------------------------- IAM PRIMARY OPTION MENU --------------------------- OPTION ===> I I - Allocate (DEFINE) a new IAM Dataset V - Allocate (DEFINE) a new VSAM Cluster D - Delete a Dataset, Cluster, Path, or Alternate Index C - Copy an IAM Dataset, VSAM KSDS, or VSAM ESDS M - Move an IAM Dataset, VSAM KSDS, or VSAM ESDS R - Rename a Dataset, Cluster, Path, or Alternate Index U - Invoke an IAM Utility program blank - Display IAM Dataset or VSAM Cluster Information Enter dataset name (required for all options except U) Dataset Name ===> myiam.cluster Dataset Type ===> C C=Cluster X=AIX P=Path (options I, V Only) Enter Model or New dsname (optional for options I and V, required for option R) Model|Newname ===> Delete Confirmation ===> YES Yes|No View IDCAMS statements ===> NO Yes|No|Only


46

Summary

IAM has evolved over the last three decades and continues to offer outstanding performance and reliability for VSAM-based applications. This evolution and continued development of the product is best illustrated by the recent introduction of IAM/RLS and IAM/PLEX to address the ever- changing business requirements of our customers.

Despite numerous hardware and software changes that have occurred over time, IAM continues to outperform VSAM. Even when taking into account enhancements such as VSAM SMB, VSAM LSR, hardware compression and extended format files, the IAM file structure still provides better performance than VSAM.

We know from the regular feedback supplied to us though our world-wide customer base that IAM still provides significant savings over VSAM KSDSs, RRDS and ESDSs in the following areas:

• DASD Space • I/Os • CPU

And these savings result in:

• Reduced Batch Elapsed times • Reduced BACKUP/REORG times • Reduced Online Response times • Improved DASD utilization

With IAM, you can be safe in the knowledge that your application data is being stored and used as efficiently as possible, with the minimum of resources required (DASD, CPU, I/O).

You are relieved of the effort required to manually tune, implement and monitor your Buffer allocations and the exposure to getting those allocations wrong. Your online system should perform better and are available each morning, because your overnight batch times are reduced to allow you to complete jobs in the window available.

IAM is the only real solution to your VSAM problems, because it addresses the root cause–the VSAM file structure–rather than trying to work around it.


47

PART 4 – File Reorganization This section describes IAM Dynamic Reorganization and other reorganization util ities included with IAM.

Introduction

IAM Dynamic Reorganization is available in IAM starting with release 9.4.01. Dynamic Reorganization provides enhanced reorganization processing for IAM Enhanced format data sets. The goal of IAM Dynamic Reorganization is to provide increased data availability for IAM data sets by reducing the time that the data is not available due to data set reorganizations. The functions and features included with IAM Dynamic Reorg are:

• Dynamic Reorganization process to perform the bulk of reorganization process while the IAM data set remains completely available to CICS online transaction processing. The data set may also be under IAM/RLS control.

• Employment of various techniques to enhance the performance of the reorganization process with use of the latest IBM Z features including 64-bit storage, z/Architecture instructions, and z/HPF I/O. Unlike typical reorganizations that use an intermediary file, the data is written directly into a new IAM data set. An optional flat file copy of the data set can be concurrently created by this process for backup purposes and optional use of z/EDC to reduce the size.

• Elimination of the need to decompress and compress data for data sets that use any form of IAM data compression. This feature provides for a very significant improvement in both CPU time and clock time to reorganize hardware compressed IAM data sets.

• Export and Import functions are available that create a flat file image of the IAM data set that has the compressed data, the file attributes for subsequent import, and the hardware compression dictionary for the data set if applicable. This is an excellent alternative to using IDCAMS REPRO for application data set backups. The exported data set can be compressed with z/EDC to further reduce the amount of space needed for the backup copy.

There are two separate executable batch utility programs:

• IAMDRCC – that facilitates the dynamic reorganization process for files that are open under CICS

• IAMSRORG – that performs the reorganization technique for files that are idle (not in use) and includes REORG, EXPORT, and IMPORT processing

These functions support IAM data sets that have up to a maximum internal IAM record length of 32,755 bytes. Compatible format IAM data sets are not supported.


48

IAMDRCC – Dynamic Reorganization

The IAM Dynamic Reorganization process improves data availability by enabling a major portion of the data set reorganization process to be performed concurrently with online CICS region updating of the data set. The reorganization process runs as a batch job and is designed to be performed automatically without any need for human interaction. Dynamic Reorganization can optionally be performed with manual control via WTOR prompts if desired. A summary of the Dynamic Reorganization process is as follows:

1. Establishes communication with CICS region and IAM/RLS region if applicable that has the specified IAM data set opened for update, and set up a mechanism to track any changed, deleted, or inserted records. IAM/RLS using batch jobs cannot have the data set being Reorged open for update processing while the IAM Dynamic Reorg is running and this is verified.

2. Calls the IAMSRORG reorganization copy program to make a copy of the IAM file as it is being used under CICS into a new IAM data set specified by the TODSN parameter.

3. As that copy is being performed, CICS continues with normal update processing of the IAM data set, while IAM tracks any updates that are taking place.

4. When the IAMSRORG copy function has completed, Dynamic Reorganization does an initial update phase by applying the tracked changes to the new reorganized copy of the file.

5. Optionally, activate two CICS GLUE Exits that cause any transaction Reads and Writes to the reorganized file(s) across process #6 (below) to tolerate the file Close, Rename, and Open. This is done by delaying any Read or Write activity that would get caught in the item #6 process, such that any active I/O still receives a zero CICS response code and all I/O to the reorganized file(s) are satisfied for the entire duration of the Dynamic Reorg running.

6. Close the file under CICS, then apply any additional updates to the new copy of the data set, do the renames, then signal CICS to re-open the data sets to resume processing.

7. Deactivate the two CICS GLUE Exits described in process #5 (above) if applicable. 8. The backup of the data set is in the renamed original data set. This can be copied if desired

into a flat file with the IAMSRORG program EXPORT function or FDRREORG for example.

IAMSRORG uses various optimization techniques to perform the reorganization copy quickly. While it is important to complete that phase quickly the critical time period is the brief period the file is closed for Step 6 above to complete the updates, rename the data sets, and re-open the data set to CICS, which is expected to complete in a few seconds.


49

Performance Example

A test was performed in our laboratory on a z/13s processor using a base cluster with an alternate index with the following characteristics:

• Record Length of 325 bytes • Key Length of 43 bytes • Alternate Key Length of 52 bytes • Loaded with 9,000,000 records • Then inserted 500,000 records prior to CICS processing • At time of reorganization, it had 9,508,792 records

There were 4 different tests run as follows with time unavailable to CICS:

• IDCAMS with VSAM data set using SMB - 7 Minutes 55 Seconds • IDCAMS with an IAM data set - 2 Minutes 38 Seconds • IAM Special Reorganization of IAM data set CICS - 23 Seconds • Dynamic Reorganization of IAM data set while open to CICS - 2 Seconds

The results below indicate the time that the data set was unavailable to CICS for processing in seconds:

Concurrent Reorg Performance

Additional tests were performed to determine the benefit of the new concurrent reorganization process. Three VSAM KSDS files were reorganized using standard IDCAMS reorganization process with three jobs running concurrently. Then the three files as IAM were reorganized


50

with the Dynamic Reorganization concurrent processing with 3 subtasks. The attributes of each file were:

• File 1: 3,003,677 records RL=200 KL=41 • File 2: 3,003,188 records RL=322 KL=42 • File 3: 9,508,792 records RL=325 KL=43 with an AIX

The VSAM batch jobs, which were all started at the same time, were completed in a best time of 8 minutes, plus 23 seconds, for a total of 503 seconds. To that time would need to be added the time to close the files under CICS plus the time to re-open the data sets.

For the IAM reorganizations, one job ran each reorganization as a subtask. The processing time for this job includes the automatic open and closing of the data sets as part of the reorganization process. The best time for the concurrent reorganizations with IAMDRCC was 53 seconds. The time the data set was unavailable to CICS was 3 seconds!

The reorganization of the three files by IAMDRCC ran 90% faster than the jobs performing the standard IDCAMS VSAM reorganization process. For IAM, this included the times to automatically close and reopen the data sets when needed. The files were unavailable to CICS for 3 seconds for a 99.5% reduction in the time that the IAM files were unavailable compared to VSAM using IDCAMS to reorganize the three data sets.

IAMDRCC – CICS Customization Requirements

CICS Customization for Dynamic Reorg

The Dynamic Reorganization process requires some customization to CICS regions that have the IAM files that are going to be reorganized by the Dynamic Reorganization process. Among the requirements are to define the IAM transaction and program that are used by the Dynamic Reorganization process and the definitions to use the EXCI interface if they have not been already defined.

The IAM Installation Control Library (ICL) contains member IAMDRORG to help you define the necessary CICS resources for using the IAM Dynamic Reorg Utility. The IAMDRORG member contains CICS DFHCSDUP batch utility statements for defining the required resources. The IAMDRORG ICL member also contains comment statements to help guide you through the resource definition process.

The IAM Dynamic Reorg Utility makes use of the CICS EXCI batch job communication interface. If the CICS region that you plan on using the Dynamic Reorg Utility with already has batch EXCI resources setup, then you will already have some of the required resources defined. The ICL member IAMDRORG comment statements advises you accordingly in this regard.


51

At a minimum you’ll need to define the following:

• The IAM IDRS transaction that invokes program DFHMIRS, which has an assigned CICS Profile that has the INBFMH(ALL) attribute

• The IAM Dynamic Reorg Server program- IAMDRCS • Add the IAM hlq.LOAD to your DFHRPL concatenation in your CICS startup JCL.

At a maximum you will also need to define the following:

• A CICS Profile with the INBFMH(ALL) attribute • An EXCI Connection definition • An EXCI Sessions definition

IAMSRORG – IAM Special Reorganization Process

IAMSRORG uses a variety of techniques to enhance the reorganization process performance resulting in a reduction of time that a data set is unavailable for processing. IAMSRORG utilizes an IAM data set reorganization process that copies data from an input IAM data set, and writes directly into an output IAM data set, without reading the index from the source data set. The program will read through the extended overflow blocks passing the records to the system SORT program to sort the overflow records in ascending key sequence. On the output side it will merge the sorted overflow records with the already in sequence prime data records and write those records to the output data set. This process is invoked by the Dynamic Reorganization IAMDRCC program to create the reorganized copy of the IAM data set as described in the prior sections. The process can also be used as a separate batch utility program to do a high performance reorganization of IAM files reducing the time that the IAM data set is not available compared to other reorganization techniques.

An optional sequential output data set can be produced that is a portable copy of the data set which can be z/EDC compressed. That portable copy of the data set contains additional information about the data set, including all of the information necessary to define the data set as part of an IMPORT process. If the original data set contained compressed data, then the portable copy will also have the compressed data. If the original data set was hardware compressed, the portable copy will contain the hardware compression dictionary needed to decompress and compress data for that IAM data set. The exported the output data set can only be utilized by the IAMSRORG IMPORT function to restore the data set to its original content. For a sequential copy of only the data in the file as the backup output from the REORG, then specification of the NOEXPORT keyword on the REORG command will produce such a copy that can be used by applications and REPRO to process the uncompressed data records that will be in ascending key sequence.


52

The functions provided by IAMSRORG are:

• REORG — Reorganize the input IAM data set directly into another IAM data set. The output IAM data set will be a reorganized data set with no overflow records. An optional sequential portable copy of the data can also be created.

• EXPORT — Create a portable copy of the input IAM data set. • IMPORT — Create an IAM data set from the portable copy created by the REORG or EXPORT

command. Has the capability to define the output IAM data set with all of the original data set attributes.


53

PART 5 – Appendix IAM INNOVATION ACCESS METHOD

Many z/OS application vendors make extensive use of VSAM, and shown below is a list of some of the most commonly used applications that have been successfully converted from VSAM to IAM.

VENDOR PRODUCT ADP Payroll American Management Systems (AMS) CUFS, Advantage Financial Management, ACAPS American Software DRP AON Hewitt (Prev. Cyborg) Payroll ASG (Prev. Mobius) ViewDirect Atos Origin (Prev. Sema, Schlumberger) CARDLINK, Payroll and Personnel (IPAS) Automated Financial Systems Commercial Loan Beta Systems Beta93 Blue Cross of AK FISS Medicare, A Payor BMC CMF Monitor, Mainview CICS, Control-M

CA Datacom, CA-7, CA-11, Activator, UFO, Intertest, Netman, Easytrieve, Bundl, Netmaster

CGI AMS Advantage, Commercial Line Systems (CLS) CSC (Prev. Vantage) Hogan, Cyberlife, Capsil, Vantage CSI Bank Trade Empower (Prev. Tesseract) EmpowerHR, EmpowerPAY Fair Isaac Corp Triad FIS (Fidelity National Information Systems) (Prev. Kirchman and Metavante)

Systematics, BASE2000, Installment Loan, CIF, Tesarec, Dealer Floor Plan, CSF

First Data VisionPLUS, Vision 21, Cardpac Fiserv (Prev. Checkfree) Pep Plus (PEP+), Mortgage-Serv General Dynamics (Prev. VIPS) Medicare Part B, SuperOP Auto Global CARMS H&W Computer Wizard, Mail, SYSM, SYSB2 IBM RMDS Infor (Prev. GEAC, Newtrend, Walker and Dun & Bradstreet, Trysyn) GL, AR, MSA, Millennium, Walker, Infopoint

Integral Systems General Ledger, HR/Payrol, HRMS JDA Software (Prev. Western Data Systems, Manugistics)

Compass Contract

Mackinney Software JES Queue Print (JQP) Macro4 Tracemaster Napersoft Automated Correspondence Tools (CCM Suite) Pitney Bowes (Prev. Group 1 and LPC) Finalist, Code 1, Mailstream, Mailsort Mailers Choice Retalix (Prev. OMI) Advanced Billing (ABS), Biceps, Prompt


54

Salary.com (Prev. Genesis) Payroll SEA TRMS Shaw Systems Collections Siemens Medical Systems Invision, Signature, Enterprise Active Directory (EAD) Sigma SAM (Student Aid Management) SunGuard (SCT) SIS+, HRS, FRS TrizeTTO Claimfacts Xybernet XY-CARD, XY-Claims, XY-Admin

Note: Numerous customers have also successfully converted their own “home-written” VSAM applications to use IAM files.


55

Customer Feedback

Here is just a taste of some of the feedback that we’ve received recently from our IAM customers:

We ran two separate tests in a controlled development environment. We executed the test jobs using original VSAM files first, and then re-ran the same job using the IAM files. This was done in order to eliminate the effect due to the difference in environmental set up of production and development.

Test I

In this test, the main program in the batch job is an in-house developed routine for customer record extraction from the current account system with Systematics subroutine SILINK. Most of the files used are Systematics master files. Here are the savings achieved by IAM:

• Elapse time: 70% reduction • CPU time: 25% reduction • EXCP count: 57% reduction • Disk space: 70% reduction

Test II

In the second test, the main program is in-house developed using Systematics subroutines including STACTM, STBCRM, STLOGTS. Here are the savings achieved by IAM: result is:

• Elapse time: 81% reduction • CPU time: 49% reduction • EXCP count: 57% reduction • Disk space: 73% reduction

“IAM has delivered good results by reducing the I/O count and CPU usage by 50 percent. Acquiring IAM to reduce I/O appears to be more cost-

effective than investing in hardware upgrades. The reduced I/O and CPU also support future growth and capacity needs for special processing like

year end…”


56

Benchmarks (IAM vs VSAM with SMB)

We ran some benchmark tests comparing the IAM “Turbo” option (with and without hardware compression) against VSAM with System Managed Buffering (SMB). Full details of the benchmarks are available on request, but below is a summary of some of the test results.

Test #1: Load file (2,000,000 Records, CI Size 8192 and a Keylength of 8)

Results without Compression

• IAM used 39% to 41% less CPU time • IAM ran in 46% to 62% less elapsed time • IAM used 85% less EXCP’s • IAM used 8% to 17% less DASD space

Results with Compression

• IAM used 1% to 35% less CPU time • IAM ran in 27% to 40% less elapsed time • IAM used 87% to 91% less EXCP’s • IAM used 25% to 45% less DASD space

“We ran a series of tests with IDCAMS (dump/reload, etc.) comparing performance between IAM and VSAM. Over the 5 separate tests IAM

demonstrated a saving compared to VSAM of 49% on I/Os, 60% on CPU time and 58% on the batch elapsed times…

“Results on some application batch testing showed even greater savings, with IAM reducing the I/Os by 84%…!”


57

Test #2: Random Reads (200,000 Reads spread throughout 10% of file)

Both IAM and VSAM required additional buffers for large record sizes

• IAM used 20% to 27% less CPU time • IAM used 21% to 36% less elapsed time • IAM used 39% to 61% less EXCP’s

Test #3: Sequential Read (Read entire file of 2,000,000 records)

• IAM used 14% to 20% less CPU time • IAM ran in 25% to 51% less elapsed time • IAM used 67% to 79% less EXCP’s

Test #4: Random Insert (Randomly insert 38,000 records within key range of 400,000) Test Files defined with 10% CI Free Space

• IAM used 66% to 72% less CPU time • IAM ran in 67% to 85% less elapsed time • IAM used 72% to 92% less EXCP’s • For uncompressed data, IAM used 26% to 32%

less space

iam concepts & facilities guide · 2020. 7. 10. · the iam/rls feature provides record- level...

Documents