ims database solution pack: user group philadelphia 2015
TRANSCRIPT
© 2015 IBM Corporation
IMS Database Solution Pack
Overview with Autonomics
Richard Phipps IBM IMS and IMS Tools
Philadelphia IMS Regional User Group
Meeting 21 May 2015
© 2015 IBM Corporation 2
IMS Tools – Why ?
The IMS base product includes a number of basic utilities for the
administration and management of IMS and it’s underlying data bases
• Backup, Recovery, Reorganization, Performance reporting, etc.
• Sufficient for some customers with lesser demands from IMS data
For many IBM customers, the ‘business critical’ nature of IMS data
demands more from what IMS utilities provide.
IMS Tools are needed for:
• Better Performance
• Enriched Features and Functions
• Higher Data Availability
• Integrated Capabilities
• Ease of Use
“Doing more with less” … this is the role of IMS Tools
© 2015 IBM Corporation 3
IMS Tools from IBM
Providing Modernization and Autonomics
IMS Fast Path Solution Pack
Autonomics IMS HP Fast Path Utilities IMS DIB Repair Facility IMS HP Image Copy IMS Library Integrity Utilities
IMS Recovery Solution Pack
IMS HP Image Copy IMS Database Recovery Facility IMS HP Change Accumulation IMS Index Builder IMS DRF Extended Functions
IMS Performance Solution Pack IMS Connect Extensions IMS Performance Analyzer IMS Problem Investigator
IMS Database Solution Pack Autonomics IMS DB Reorganization Expert - Unload, Load, Index Build, Prefix Resolution/Update IMS HP Image Copy IMS HP Pointer Checker IMS Library Integrity Utilities IMS HALDB Toolkit IMS Online Reorg Facility
IMS Sequential Randomizer Generator Data Base Administration
IMS Cloning Tool IMS Database Control Suite
Utility Management
IMS HP Image Copy IMS DEDB Fast Recovery IMS Recovery Expert
Backup and Recovery Batch Terminal Simulator
Program Restart Facility Application Management
Performance Management
IBM Transaction Analysis Workbench IMS Buffer Pool Analyzer IMS Network Compression Facility
System / TM Administration
System IMS Command Control Facility IMS ETO Support IMS HP Sysgen Tools IMS Queue Control Facility IMS Workload Router TM IMS Configuration Manager IMS Sysplex Manager
Regulatory Compliance
Guardium ITAP for IMS IBM Infosphere Guardium Data Encryption for DB2 and IMS Databases
IBM Tools Base for z/OS
© 2015 IBM Corporation 4
Today - Typical Concerns and Challenges
Increased use of data assets - Bigger database size - Higher demands for continuous data availability
New business requirements - Enabling host data for access outside of the zSeries (host) platform - New governmental regulations
Cost containment - Reduce CPU, I/O, and elapsed execution times - Budgetary Concerns
Support staff concerns - Head count reductions - Retirements - Experience and knowledge level
The “shrinking skill set” - A top concern for all
© 2015 IBM Corporation 5
The “Shrinking Skill Set” – Some observations
Issue: The “z” support community is getting older which creates an impending skills
problem that IT organizations are already beginning to deal with Facts:
80% of IMS DBAs are over the age of 55 Within the next 10 years, this entire group will be moving on to retirement
Most companies have IMS DBAs with 25+ years of experience
Estimates are it takes 5-7 years for someone to become a truly effective DBA
Industry statistics show that on average, new employees will remain on the same job for between 3-5 years
Response:
IT organizations have some “big shoes to fill”
Provide tools that can capture the wealth of experience
Put new tools into place to accelerate the learning curve for next generation IT support people
© 2015 IBM Corporation 6
IMS Automated Database Solutions
Checks data base status on a regular basis
User specified thresholds for key indicators
Performs reorganization only when necessary
Performs auxiliary functions: IC, PC, IB, etc.
Keeps data bases performing optimally
Saves human resources
Saves computer resources
Cost efficient solution for both predictable and unpredictable data base growth and
activity
© 2015 IBM Corporation 7
“No charge” value-added products to support
extended use of IBM data management tools
** Capabilities designed to address
concerns of a “shrinking skill set” Provides IMS database compression tool to address
needs of growing IMS database sizes
GUI to provide detailed insight into the health and
availability of IMS databases through an easily
accessible Web interface
Autonomics capability to implement the automation of
tools execution using detailed understanding of the
current state of the IMS databases
Report repository of historical execution of IMS
utilities used in managing the IMS databases
IBM Tools Base
IMS Tools provided:
IMS Hardware Database
Compression Extended
Autonomics Director for
IMS & DB2
IMS Tools Knowledge Base
…and More
© 2015 IBM Corporation 8
IMS Hardware Compression Extended
Productivity aid for implementing Hardware Assisted Data Compression
Functions:
Extends IMS basic HDC support with utilities that ease compression
implementation and provide additional flexibility and function
Works on image copies, High Performance Unload files and IMS unload
files
Includes sample Compression Dictionaries
Enables effectiveness of existing dictionaries to be assessed
Allows monitoring of dictionary effectiveness over time
Builds DBD and reload JCL
IBM Tools Base
© 2015 IBM Corporation 9
IMS Tools Knowledge Base
Implement a report repository for IMS Tools
Creates a historical perspective of IMS databases for the DBAs
Reporting stored in the IMS Tools Knowledge Base can be searched, retrieved, and viewed effortlessly • Through ISPF interface or IMS Admin Console
Facilitates communication between IMS Tools products
Tools Registration
Environment Information
Sensor Data
Policies
Autonomics Engine
IBM Tools Base
© 2015 IBM Corporation 10
IMS Tools Vision - Autonomics
Sensors collect resource statistics
Policies evaluate sensor data and identify potential problems
Automation orchestrates the collection and evaluation of sensor data
Modernization presents an interactive modern interface for managing the system
Putting information to work
Sensors Policies
Automation Modernization
Not a vision, but a reality
IBM Tools Base
© 2015 IBM Corporation 11
Sensors: Collecting the Basic Information You Need
Statistical point-in-time sensor data on your FF/FP Databases
Stored in IMS Tools Knowledge Base repository
Historically maintained per user specifications
Over 60 separate data elements related to space usage, optimization, and
fragmentation
• data set extents, DASD volume usage, data set free space, roots distribution, RAP usage, CI/CA
splits, and IMS free space, etc
Two methods of collection:
Standalone database sensor utilities for full-function and Fast Path databases
Integrated with existing IMS Tools
Integrated Tools support
High Performance Image Copy, High Performance Pointer Checker
Fast Path Analyzer, Fast Path Online Pointer Checker
Sensors
IBM Tools Base
© 2015 IBM Corporation 12
List of Full Function Sensor Data collected
Database Record Statistics (per database or HALDB partition) Nbr. of DB records Avg. DB record length
Randomizer Statistics (per HDAM or PHDAM partition) Nbr. of total RAPs Nbr. of unused RAPs % of number of unused RAPs Nbr. of synonyms
% of number of synonyms Nbr. of root not on home block % of root not on home block % of segment data in overflow
Nbr. of roots in overflow % of number of roots in overflow Bytes of segments in RAA
Volume/Extents Statistics (per data set) Allocation type (CYL, TRK, …) Primary allocation amount Secondary allocation amount SMS-managed or not
Max. nbr of extents for the d.s. Max. nbr. of extents for the volume Nbr. of extents allocated Nbr. of volumes used
Nbr. of unused volumes Nbr. of unused assigned volumes Nbr. of unused candidate volumes
Nbr. of available remaining extents determined by the max. nbr. of data set extents and the max. nbr. of extents available on volumes assigned to the data set
Data Set Space Usage Statistics (per data set) Block/CI size Nbr. of blocks/CIs used Max. size of the data set % of data set size against the max.
High-Allocated-RBA High-Used-RBA
IMS Space Utilization Statistics (per data set) Total bytes of segment data Total bytes of free spaces Total bytes of slack bytes % of free spaces
% of segment data % of unused bytes in the data set Total nbr. of segments Total nbr. of VL segments
Total nbr. of VL-split segments % of nbr. of VL-split segments Total nbr. of slack bytes Avg. nbr. of slack bytes per block
Total nbr. of FSEs Avg. nbr. of FSEs per block Nbr. of FSEs valid for shortest segments Nbr. of FSEs valid for longest segments
Avg. nbr. of non-reusable FSEs Total nbr. of pointers Total nbr. of ptrs pointing external block % of nbr. Of ptrs pointing ext. block
HISAM/SHISAM Statistics (for HISAM) Logical record length Total nbr. of CI splits % of nbr. of CI splits Total nbr. of CA splits
% of nbr. of CA splits Total nbr. of HISAM delete bytes % of nbr. of HISAM delete bytes
Sensors
IBM Tools Base
© 2015 IBM Corporation 13
Policies: Using Sensor Data to Make Decisions
Policy definitions are used to evaluate specific database states
Threshold values are compared against sensor data for a given database or group of databases
When thresholds are met or exceeded, exceptions occur
Works “out of the box”
Ships with predefined policies and threshold values
Full ISPF interface provided for policy management
Customizable to fit your shop
You can define your own sets of threshold values
Customize the messages sent when exceptions do occur
Specify who receives which messages and how • WTO, e-mail, or text
Policies
IBM Tools Base
© 2015 IBM Corporation 14
Autonomics Director for IMS & DB2
IBM Tools Autonomics Director (Passive)
Automates collection and analysis of Sensor Data
Recommends when databases should be reorganized
• With email or text notifications
Provides a scheduling feature that allows you to control how frequently sensor data is
collected and how frequently policies are evaluated
Flexible scheduling around pre-defined PEAK times
IBM Tools Autonomic Director (Active)
Actively initiate recommended actions on user-defined database groups • Discovery feature for identifying related database groups
• Ability to manage and coordinate reorganization of multiple IMS database groups as if
reorganizing a single database
• Flexible scheduling only in pre-defined Maintenance windows
Automation
IBM Tools Base
Help prevent problems before they occur – Higher availability and performance through optimally organized data
Consistent and reliable monitoring of database status
– Regular collection/evaluation at predefined intervals
© 2015 IBM Corporation 15
© 2014 IBM 15
Changing the “face” of the IMS platform and
Management of System z
IBM Management Console for IMS & DB2 Databases for z/OS
Modernization Modernization
© 2015 IBM Corporation 16
Provide a single holistic, easy-to-use interface to manage IMS/DB2 systems and
databases
Consolidate information from various tools to paint a more complete picture of IMS/DB2
systems and databases
Leverage the latest web technologies for a richer user experience
Access from anywhere via the Internet using standard web browsers
Prepare the next generation of DBMS DBAs and System Programmers
through an integrated, context sensitive help system
IBM Management Console for IMS & DB2 Databases for z/OS
© 2015 IBM Corporation 17
Object Health
and Autonomics Enterprise-wide
Navigation
Integrated Help
Graphical Visualization
of data not possible in ISPF
Provides a single, holistic easy-to-use interface to manage IMS and DB2
Zero-install web-based interface
Consolidate information from IMS, DB2 and tools from across the entire enterprise
Reduced time for problem identification and resolution through tight integration with IMS and DB2 Autonomics
Dramatically reduced learning curve for new users of IMS and DB2
Now a separate product available no-charge (5655-TAC)
Extensible by growing number of products and solution packs adding additional value
IMS HP
Tools
Autonomics
Director Discovery
Sensors
IBM Management Console for IMS & DB2 Databases for z/OS
© 2015 IBM Corporation 18
Drill down from the IMSPLEX level through:
IMSPLEX members
IMS Connect
IMS Subsystems
Transactions
Programs
Routing Code
Databases
Identify, stop, start IMS resources
Explore resource relationships
OM Command Builder
IBM Management Console for IMS & DB2 Databases for z/OS
© 2015 IBM Corporation 19
Graphical control of IMS
Autonomics Director
Define monitor lists, setup
peak times and
maintenance windows
Web access to IMS reports
from a variety of Tools
Search and filter by type,
date, completion code, etc
ITKB Reports
Autonomics Control
IBM Management Console for IMS & DB2 Databases for z/OS
© 2015 IBM Corporation 20
Extended IMS Autonomics and
Database information
Easily identify and diagnose
symptoms and recommended
actions through IMS Database
Autonomics
Identify trends and make projections
through database sensor charts
Visualize IMS FF/FP Databases
through Library Integrity Utilities
LIU Reports
Extended with the IMS FF/FP Solution Pack
IBM Management Console for IMS & DB2 Databases for z/OS
© 2015 IBM Corporation 21
Progressive drill down through
variety of DB2 object dashboards
Autonomics Director for DB2 for
z/OS (in the no-charge Tools
Base) enables:
Charting of DB2 object statistics
through RTS snapshots with
Autonomics control to define
profiles and maintenance
windows
Integrated support for the DB2
Admin Task Scheduler
DB2 Subsystem
Maintenance Windows
IBM Management Console for IMS & DB2 Databases for z/OS
© 2015 IBM Corporation 22
Identification and Diagnosis of
symptoms and recommended
actions for REORGs, ICs and
Runstats
Reporting on historical utility
execution including timestamp,
elapse time, system output, etc.
Graphical interfaces to define
Automation Tool Object, Utility,
Exception, and Job Profiles
Extended with the DB2 Utility Solution Pack
IBM Management Console for IMS & DB2 Databases for z/OS
© 2015 IBM Corporation 23
Quickly identify and drill down to databases and objects that need your attention from a single starting point
Easily manage by exception and recommendation, taking action before problems occur
Rapidly interpret statistical trends to verify and project
Shorten the learning curve for new administrators
…all from a unified IMS and DB2 interface
IBM Management Console for IMS & DB2 Databases for z/OS
© 2015 IBM Corporation 24
© 2014 IBM 24
Our Vision – Putting information to work for you
“No charge” systems and data
management solutions from IBM
This is critical technology that we
want all our customers to have
and use now and into the future
Sensors Policies
Automation Modernization
© 2015 IBM Corporation
© 2015 IBM Corporation 25
Designed to reduce technical expertise by providing
an integrated, high performance solution
Provides a complete set of high performance utilities to unload,
load, reorganize, verify, and report on Full Function and HALDB
databases
Functions to assist DBAs in identifying and performing the tasks
of maintaining the health, performance, and recoverability of IMS
databases
Features for the 'smart' reorganization of IMS databases that
can help improve system availability
IMS Database Solution Pack for z/OS
IMS Tools provided: • IMS Online Reorg Facility
• IMS DB Reorg Expert
• HP Unload
• HP Load
• IMS Index Builder
• HP Prefix Resolution
• HP Image Copy
• HP Pointer Checker
• IMS Library Integrity Utilities
• HALDB Toolkit
Utilities to validate, compare, map, recover, and report on IMS libraries
Provides tool for fast backup of IMS databases
Simplified use of IMS data management processes through tool integration
© 2015 IBM Corporation 26
IMS Database Reorganization Today – How are you doing it now?
Workload for data management increases as your business grows
Increasing volume of repetitive tasks such as free space defrag
Processing efficiency is becoming important
More frequent needs for database logical/physical structure changes
Deep IMS DBA skills are needed for these complex tasks
Plan/Do Actions Diagnose Databases
Pointer
Checker
Reports % of segment splits
Avg nbr of FSEs
% of max. dataset size
Number of extents
Statistics Values
. . .
Diagnosis Result
Exception 1
Exception 4
. . .
Each may be simple,
but the total work volume
is getting larger
. . .
Processing efficiency
is becoming important
How to do it
this time?
When can
it be done?
Deep IMS DBA
skills are needed
in planning actions
Exception 2
Exception 3
Check Key
Stats Items
IMS DBA
Simple Repetitive Actions
(e.g. free space defrag)
Complex Actions
(e.g. Structure Change)
© 2015 IBM Corporation 27
IMS Database Reorganization Expert
Provides a capability to perform a database reorganization while the database
remains in read only status
Provides automated one step batch job process to:
Reorganize IMS databases to shadow datasets
Rebuild the indexes
Perform image copy after the reorg with hash pointer checking
Rename shadow datasets to original production name
Perform automated /DBR and /STA
© 2015 IBM Corporation 28
IMS Online Reorganization Facility
Provides a capability to perform a database reorganization while the database
remains online and available for reads and updates
Designed to address customer needs for 24x7 continuous data availability
Provides automated one step batch job process to:
• Reorganize IMS databases to shadow datasets
• Rebuild the indexes
• Perform image copy after the reorg
• Capture changes to the original datasets
• Apply captured changes to the shadow datasets
• Rename shadow datasets to original production name
• Perform automated /DBR and /STA
© 2015 IBM Corporation 29
Conditional Reorganization – Only reorg what needs it!
Conditional Reorganization Process Steps
DB Administrator submits conditional reorganization job
Reorganization utility collects database statistics and stores them in repository
• Tools Base (Policy Services) evaluates the statistics by predefined policies
Reorganization utility reorganizes the database only when it is necessary
Reorganization utility generates diagnosis report and stores it in repository
DB Solution Pack
Reorganization Utility
• DB Reorg Expert
• Online Reorg Facility
TSO
Submit REORG job
IBM Tools Base
IMS Tools Knowledge Base
Policy
Repository
Store reports
Statistics
Evaluation
Results Diagnosis Report
© 2015 IBM Corporation 30
Specifying a Conditional Reorg Mode of Operation
The following modes are supported in both DB Reorg Expert and ORF:
Conditional Reorganization mode
• Database is reorganized only when necessary.
• DB Reorg Expert: CONDREORG=YES
• ORF: CONDREORG(Y)
Reorganization and Diagnosis mode
• Database is always reorganized and Diagnosis report is generated.
• DB Reorg Expert: CONDREORG=YES,REORGDIAG
• ORF: CONDREORG(Y,REORGDIAG)
Diagnosis-only mode
• Database is not reorganized. DB statistics are collected and evaluated.
• DB Reorg Expert: CONDREORG=YES,DIAGONLY
• ORF: CONDREORG(Y,DIAGONLY)
© 2015 IBM Corporation 31
Conditional Online Reorganization Work Flow
Benefits of Conditional Online Reorganization
A database can be reorganized only when it is really needed
The Diagnosis Report provides an evidence of the necessity of reorganization and
effectiveness of the reorganization
IMS Online Reorganization Facility
Dia
gn
os
is o
f O
nli
ne
DB
Renamed to
original name Renamed to
original name
Renamed to
original name Renamed to
original name
Log R
ecovery
Pro
cessin
g
Shadow DBD Shadow DBDS
Index Index Shadow Index
IC
IC IC
DB is Offline
NO
TIF
Y.R
EO
RG
DB
DS
Nam
e S
wappin
g
CH
AN
GE
.DB
AU
TH
RE
AD
OF
F
Co
py T
as
k
NO
TIF
Y.I
C/U
IC
Resta
rtin
g D
B
/STO DB
&
/STA DB
/STA DB
Online Online
Original Index Original Index
Original Index
Original DBDS Original DBDS Renamed to
original name
Online Reorganization Workflow Control (used only when reorg needed)
Sta
rt L
og C
aptu
re
Reorganization need is checked
z/OS consoles (and/or TSO users)
Detected or
remaining exceptions
are notified
(based on the policy)
Un
loa
d T
as
k
/STO DB
&
/STA DB
Sw
itch to C
hange C
aptu
re
Unload
Data Set
Relo
ad
Ta
sk
Shadow DBDS Shadow DBDS
Ind
ex
Bu
ild
Ta
sk
Ima
ge
Co
py T
as
k
(can
in
clu
de P
oin
ter
Ch
eck)
/DBR DB
Makin
g D
B O
fflin
e
Ap
ply
Ta
sk
Dia
gn
os
is o
f O
nli
ne
DB
Co
nd
itio
na
l R
eo
rg
Po
st
Pro
ce
ss
ing
Online Online
© 2015 IBM Corporation 32
Conditional Online Reorganization – Sample JCL
Three types of policy selection
By database type
Default
By database name
Used to specify a database
specific policy
By policy name
Used to specify a policy for a
group of databases
//ORFJOB EXEC PGM=HRFREORG,REGION=80M,DYNAMNBR=999 //STEPLIB DD DISP=SHR,DSN=ITB.SFOILOAD // DD DISP=SHR,DSN=ITB.SHKTLOAD // DD DISP=SHR,DSN=DBSP.SHRFLOAD // DD DISP=SHR,DSN=DBSP.SHPSLMD0 // DD DISP=SHR,DSN=IMS.SDFSRESL //IMSDALIB DD DISP=SHR,DSN=IMS.MDALIB //IMS DD DISP=SHR,DSN=IMS.DBDLIB //SYSPRINT DD SYSOUT=* //HRFSYSIN DD * REORG DBD(SAMPLEDB) - LOG1DSN(DATADS.LOG.&DBD.&DATA1.&TIME2) - IC1DSN(DATADS.IC1.&DBD.&DATA1.&TIME2) - CONDREORG(Y) - ITKBSERVER(FPQSVR00) - CR.POLICYBY(DBTYPE) /*
CR.POLICYBY(DBTYPE)
CR.POLICYBY(DBDNAME)
CR.POLICYBY(NAME) CR.POLICYNM(MY.POLICY.GROUP1)
By database type
By database name
By policy name
Policy to be selected
SYS.DBDTYPE.type
SYS.DBDNAME.dbdname
MY.POLICY.GROUP1
(type = HDAM, HIDAM, PHDAM,
PHIDAM, HISAM, or SHISAM)
Specifies Conditional Reorg Mode
Specifies XCF Group Name for
ITKB Server to connect
Specifies how a policy is to be selected
,DIAGONL(Y) -
Specifies Diagnosis Mode
© 2015 IBM Corporation 33
Database Diagnosis Report
Processing result of Smart Reorg job is summarized in the Database
Diagnosis Report
The report provides the following information:
Summary of Policy Evaluation
•Policy that was applied to the database
•Reorganization need
•Result of policy evaluation
•Result of policy evaluation after reorganization (if performed)
•Evaluation summary message (in the previous chart)
Sensor data values before and after reorganization
•Differences between the two
© 2015 IBM Corporation 34
Database Diagnosis Report - Example
Basic attributes of the subject database is printed in the title page.
IMS DB Reorg Expert - V4R1 Database Diagnosis Report Page: 1 5655-S35 Date: 12/16/2009 Time: 00.05.14 Summary of Database Definition ------------------------------- Database.......................... BKDB Database Type..................... HIDAM Data Set Organization............. VSAM Number of Data Set Groups......... 1
The following sections follow this title page:
Summary of Policy Evaluation
Sensor data values before and after reorganization
1. Database Record Statistics section
2. Randomizing Statistics section (only for HDAM and PHDAM)
3. Volume and Extent Statistics section
4. Data Set Space Usage Statistics section
5. IMS Space Utilization Statistics section
6. HISAM Statistics section (only for HISAM and SHISAM)
※ Sections 3 through 5 are printed for each data set group
© 2015 IBM Corporation 35
Database Diagnosis Report – Summary of Policy Evaluation IMS DB Reorg Expert - V4R1 Database Diagnosis Report Page: 2 5655-S35 Date: 12/16/2009 Time: 00.05.14 Summary of Policy Evaluation (DBD: BKDB ) ----------------------------------------------------------------- Name of Policy Applied............ SYS.DBDTYPE.HIDAM Policy Locale..................... Global Reorganization Need............... Yes Exceptions before Reorganization -------------------------------------------------------------------------------- The number of available extents for a data set of BKDB is small Class: DATA_SET_EXTENTS_AVAILABILITY Level: CRITICAL Rule: G:IBM.DBDS_EXTENTS.10 Threshold Set: HIGH The size of a data set in BKDB, which still has a certain amount of free space, has increased Class: GROWING_DBDS_WITH_FREE_SPACES Level: CRITICAL -> REORG Rule: G:IBM.DBDS_GROWTH.20 Threshold Set: HIGH The fragmentation of free space in BKDB has increased Class: FRAGMENTED_FREE_SPACES Level: CRITICAL -> REORG Rule: G:IBM.FRAGMENTATION.10 Threshold Set: HIGH A data set of BKDB has many pointers that point to other bocks or CIs Class: EXCESSIVE_SEGMENT_SCATTERING Level: SEVERE Rule: G:IBM.SEGM_SPREAD.10 Threshold Set: MED -------------------------------------------------------------------------------- Exceptions after Reorganization -------------------------------------------------------------------------------- BSN2800I GENERAL STATUS: RESOURCE=BKDB ACTION_NAME=REORG EXECUTION_STATUS=SUCCESSFUL The number of available extents for a data set of BKDB is small Class: DATA_SET_EXTENTS_AVAILABILITY Level: CRITICAL Rule: G:IBM.DBDS_EXTENTS.10 Threshold Set: HIGH -------------------------------------------------------------------------------- Summary of Policy Evaluation and Action: -------------------------------------------------------------------------------- BSN2904I BKDB HAS BEEN REORGANIZED, BUT IT IS STILL IN A CRITICAL STATE --------------------------------------------------------------------------------
Policy name
The locale where the
policy is defined
Result of decision on
reorganization need
Summary message, which shows that a
CRICTICAL exception still remains after
the reorganization.
Exceptions detected before the
reorganization. In this example,
Three CRITICAL exceptions were
detected
Two of them can be removed by
REORG (unload and reload)
One sever exception was detected
Exceptions that still remain after the
reorganization. In this case, one CRITICAL
exception remains.
Result of the reorganization
processing (success/failure)
1
2
3
4
1’
© 2015 IBM Corporation 36
Database Diagnosis Report – Database Record Statistics
IMS DB Reorg Expert - V4R1 Database Diagnosis Report Page: 3 5655-S35 Date: 12/16/2009 Time: 00.05.14 Database Statistics (DBD: BKDB ) -------------------------------------------------------------------------------- ============================== Database Record Statistics ============================== Note: The mark * in column P means that the data element is used in the policy. Data Element Name P Before Reorg After Reorg Difference ------------------------ - ---------------- ---------------- ------------------ DB_NUM_ROOT * 8,560,000 8,560,000 0 DB_AVG_DBREC_LENGTH * 288.06 288.06 0
The number of
database records
Average database
record length
© 2015 IBM Corporation 37
Database Diagnosis Report – Volume/Extent Statistics ... Data Set Statistics (DBD: BKDB , DSG: 01) -------------------------------------------------------------------------------- ============================ Volume/Extent Statistics ============================ Note: The mark * in column P means that the data element is used in the policy. Data set definition and environment information ================================================= Data Element Name P Before Reorg After Reorg Difference ------------------------ - ---------------- ---------------- ------------------ DB_FLAG_SPACE_TYPE C C No change DB_NUM_PRI_SPACE 1,000 1,000 0 DB_NUM_SEC_SPACE 100 100 0 DB_FLAG_SMS * Y Y No change DB_MAX_EXT_DS 251 251 0 DB_MAX_EXT_VOL 123 123 0 Data set usage information ============================ Data Element Name P Before Reorg After Reorg Difference ------------------------ - ---------------- ---------------- ------------------ DB_NUM_EXT 37 32 -5 DB_NUM_VOL 3 3 0 DB_NUM_UNUSED_VOL 0 0 0 DB_NUM_UNUSED_VOL_SER 0 0 0 DB_NUM_UNUSED_VOL_CAND * 0 0 0 Remaining available data set extent estimation ================================================ Data Element Name P Before Reorg After Reorg Difference ------------------------ - ---------------- ---------------- ------------------ DB_AVAIL_EXT_LESS_100 * Y Y No change DB_NUM_AVAIL_EXT * 0 11 +11 DB_AVAIL_EXT_LIMIT VOL_FREE_EXTENTS VOL_FREE_EXTENTS No change Remark: If DB_NUM_UNUSED_VOL_CAND is not zero, more extents than those indicated by DB_NUM_AVAIL_EXT might be available.
Information on extents and used/unused
volumes for the subject data set
Estimation for extent availability
Information on data set allocation
parameters and DFSMS environment
© 2015 IBM Corporation 38
Database Diagnosis Report – Data Set Space Usage Statistics
IMS DB Reorg Expert - V4R1 Database Diagnosis Report Page: 5 5655-S35 Date: 12/16/2009 Time: 00.05.14 Data Set Statistics (DBD: BKDB , DSG: 01) -------------------------------------------------------------------------------- =================================== Data Set Space Usage Statistics =================================== Note: The mark * in column P means that the data element is used in the policy. Data Element Name P Before Reorg After Reorg Difference ------------------------ - ---------------- ---------------- ------------------ DB_BLOCK_SIZE 4,096 4,096 0 DB_NUM_DBDS_BLOCKS 999,179 673,780 -325,399 DB_MAX_DS_SIZE 4G 4G 0G DB_PCT_OF_MAX_DS_SIZE * 96% 65% -31 DB_RBA_HIGH_ALLOC 4,128,768,000 2,801,664,000 -1,327,104,000 DB_RBA_HIGH_USED 4,092,641,280 2,759,806,976 -1,332,834,304
This shows that, within data elements in
this category, only the value for the data
element DB_PCT_OF_MAX_DS_SIZE is
referrred to in one or more rules defined in
the selected policy.
You can see that the size of data set is
reduced by the reorganization.
© 2015 IBM Corporation 39
Database Diagnosis Report – IMS Space Utilization Statistics
IMS DB Reorg Expert - V4R1 Database Diagnosis Report Page: 6 5655-S35 Date: 12/16/2009 Time: 00.05.14 Data Set Statistics (DBD: BKDB , DSG: 01) -------------------------------------------------------------------------------- ==================================== IMS Space Utilization Statistics ==================================== Note: The mark * in column P means that the data element is used in the policy. Data Element Name P Before Reorg After Reorg Difference ------------------------ - ---------------- ---------------- ------------------ DB_BYTES_SEG 2,465,800,000 2,465,800,000 0 DB_BYTES_FREE_SPACE 1,610,723,680 283,136,678 -1,327,587,002 DB_BYTES_UNIDENTIFIED * 110 0 -110 DB_PCT_BYTES_FREE_SPACE * 39% 10% -29 DB_PCT_BYTES_SEG * 60% 89% +29 DB_PCT_UNUSED_BYTES * 1% 1% 0 DB_NUM_SEG 85,620,000 85,620,000 0 DB_NUM_VLSEG 8,560,000 8,560,000 0 DB_NUM_VLSEG_SPLIT * 0 0 0 DB_PCT_NUM_VLSEG_SPLIT * 0% 0% 0 DB_NUM_UNIDENTIFIED * 51 0 -51 DB_AVG_NUM_UNIDENTIFIED * 0 0 0 DB_NUM_FSE * 23,728,638 673,759 -23,054,879 DB_AVG_NUM_FSE * 23.75 1.00 -22.75 DB_NUM_FSE_MIN * 23,558,660 673,759 -22,884,901 DB_NUM_FSE_MAX * 718,662 673,759 -44,903 DB_AVG_NUM_NOREUSE_FSE * .17 0 -.17 DB_NUM_PTR 85,070,408 84,946,241 -124,167 DB_NUM_PTR_DIFF_BLK * 32,424,879 663,666 -31,761,213 DB_PCT_NUM_PTR_DIFF_BLK * 38% 1% -37
Free spaces fragmentation on each
database block has been resolved by
unload and reload of the database.
Segments of a database record fit in a
single database block in almost all
database records.
Allocated space is more efficiently used by
segment data.
© 2015 IBM Corporation 40
IMS High Performance Unload
IMS High Performance Unload (HP Unload) provides high performance database
retrieval service called High Speed Sequential Retrieval (HSSR) Engine
Provides DL/I call analyzer, segment retriever, and buffer handler that are optimized for GN
calls
• Similar to OSAM Sequential Buffering, but HSSR Engine supports VSAM ESDS also
Supports various unload record formats (HD, SH, CS, CP, F1, F2, F3, F6, A1, A2, A3, or A4)
Supports user-defined unload record formats
Provides an Unload API
• DL/I call and EXEC DLI commands: GU, GHU, GN and GHN call
HP Unload provides two unload utilities (FABHURG1 and FABHFSU)
HP Unload provides “Unload API”, which enables user DL/I unload applications to run on HP Unload without modification
IMS DLIBATCH Native DL/I Call Handler
Native Buffer Handler
IMS
HD Unload
utility
User
Unload
Application
User
Unload
Application
IMS HP Unload
Unload
utility
FABHURG1
JCL & output compatibility
HSSR Engine
No change is needed in many cases
© 2015 IBM Corporation 41
HP Unload – Major features
Common to FABHURG1 and FABHFSU utilities
Can unload a database without decompressing compressed segments
─This can decrease the CPU time and elapsed time
Supports multiple pre-defined unload record formats other than the
standard HD format
Can use HSSR Engine’s optional features to be explained later
IMS High Performance Unload
© 2015 IBM Corporation 42
HP Unload – FABHURG1 unload utility
Is a simple alternative for IMS HD Reorganization Unload
utility
Can run with using the same JCL statements
Can run on DLI and DBB regions in addition to ULU region
Supports a system exit for function extensions:
An exit routine name is specified in SYSIN
IBM-provided exit routines:
• FABHEXTR
• Creating a database extract for database tuning experiments or testing
• FABHKEYX
• Separating output data set at migration unload based on a set of high key
values specified
Users can write their own exit routines to add functions
IMS High Performance Unload
© 2015 IBM Corporation 43
HP Unload – FABHFSU unload utility
Is an unload utility for special purposes Uses JCL statements similar to IMS DLIBATCH
Supports not only ULU region, but also DLI region with a PSB having multiple PCBs for multiple databases
• Sensitivity can be controlled by using PCBs
Can produce multiple outputs with different formats for each output by a single run
Additional segment editing can be done by using user exit routines Different user exit routine can be specified for each output
Provides Parallel Scan Facility • For the compatibility with the old product FSU II
IMS High Performance Unload
© 2015 IBM Corporation 44
HP Unload – Unload API
Optimized for GN and
GNP calls with an
unqualified SSA
Enables an IMS DL/I batch
application program, which
reads a database
sequentially, to use HSSR
Engine without being
recompiled or relink-edited
Supports DL/I calls and
EXEC DLI commands
Through this API, the application program can retrieve and process
a large number of segments efficiently
IMS High Performance Unload
© 2015 IBM Corporation 45
HP Unload – HSSR Engine’s optional features
Report Database Tuning Statistics
Provides information about:
• the need for database reorganization
• the quality of HDAM or PHDAM randomizing
Report information on long database records
DB records that have the following characteristics:
• DB records whose length are greater than the specified bytes
• DB records that would require more than the specified number of I/Os for
retrieval
Information written for each DB record:
• The length of the DB record
• The number of database I/Os required to read all database segments of the
DB record
• The key value of the root segment of the DB record
IMS High Performance Unload
© 2015 IBM Corporation 46
HP Unload – HSSR Engine’s other features
Ability to continue processing after sequence errors Optional sequence-key checks for twin chains
Three options on an error: • ABEND
• Returns a GG status code and not unload the segment containing the error, or its dependents
• Returns a GX status code and unloads the segment in error
Reading a corrupted database Two options:
• Bypassing pointer errors
• Force the roots of HIDAM/PHIDAM DB to be accessed from the primary index
Detailed diagnostic information can be provided for sequence errors and pointer errors
IMS High Performance Unload
© 2015 IBM Corporation 47
IMS High Performance Load
Provides a high performance database load service
Processes compressed segment data as input
If unload dataset created by HP Unload utility
Supports various input record formats, including
Formats supported by HP Unload
User-defined unload record formats produced by HP Unload utility
Generates statistics reports to aid in tuning the database
Checks for database updates in between the unload and reload to prevent data
corruption
IMS DLIBATCH Native DL/I Call Handler
Native Buffer Handler
IMS
HD Reload
utility
User
Load
Application
DB
User
Load
Application
IMS HP Load
Reload
utility
No change is needed in many cases
JCL & input compatibility
HP Load service
© 2015 IBM Corporation 48
Can initialize an empty database or partition
HDAM/PHDAM partition: the entire root addressable area is preformatted
HIDAM/PHIDAM partition: a root segment with a key of all X'FF' is inserted
Provides options for whether/how to build ILDSs for HALDB partitions
Provides a user exit facility for additional user processing of each segment
WF1 record processing
WF1 records can be split into two groups:
• Records for logical relationship maintenance
• Records for secondary index maintenance
and be written into separate outputs
WF1 record generation can be suppressed
IMS High Performance Load
© 2015 IBM Corporation 49
Sorting before reload
Segment records in the input data set can be sorted internally before the reload processing starts
With this function, the SORT step before the reload step can be eliminated when the randomizing for HDAM/PHDAM has been changed between unload and reload
The function can be activated automatically if the input data set was created by HP Unload with certain options specified
IMS HP Load provides a Load API
Load API is optimized for ISRT calls
User initial load program can run on HP Load much faster than that on IMS DL/I
• The load program must specify a PCB with PROCOPT=L or LS
• For HDAM, internal physical sequence sort will be made before actual insert
for better performance
Calls that use AIB is supported
EXEC DLI is supported
Load API reduces time and cost for the initial load process
IMS High Performance Load
© 2015 IBM Corporation 50
IMS Index Builder
Build or rebuild index databases
Full-function non-partitioned databases
Secondary indexes: multiple indexes in one job step
Primary index of HIDAM or PHIDAM partitions
HALDB Indirect List Data Sets (ILDS)
Streamlines index creation, reorganization, maintenance, and recovery.
Eliminates the need for the traditional unload and reload processes at the time of reorganization of the primary database
Eliminates the need to image copy indexes
Exploits parallel processing by overlapping and integrating processes
Using subordinate address spaces (SAS) for parallel prime data scans and
index data sorts
Loading records into each index database in parallel
Overlapping DB scan steps and the index record sort steps overlap
© 2015 IBM Corporation 51
Creating secondary indexes by using DFSURWF1/DFSURIDX as input
DFSURWF1 data set that was created by the IMS HD Reorganization Reload utility or HP Load can be used
Data needed for prefix resolution are extracted from the DFSURWF1 file and written to a new data set, improving the performance of prefix resolution
DFSURIDX data set created by IMS Prefix Resolution utility can be used
PSINDEX support
Applies parallel partition scan to HALDB
Rebuilds selected PSINDEX partitions, and offers support for using Partition Selection Exits (PSE) when rebuilding PSINDEXes.
Building primary indexes
Builds a copy of the primary index of HIDAM database using the existing primary index as input
Creates new primary indexes of HIDAM and PHIDAM databases where input is based on a scan of the physical database without using the existing primary index
Index data set allocation and initialization
Supports IDCAMS delete/define functionality for data sets being built based on a user-provided procedure library so that no preliminary IDCAMS job steps are required
Can only initialize IMS ILDS, primary, and secondary index data sets by using an empty or dummy DFSURIDX data set as input
IMS Index Builder
© 2015 IBM Corporation 52
IMS High Performance Prefix Resolution
Enables you to resolve and update prefixes of IMS databases that are involved in
logical relationships as a single job step
Eliminates the WF2 and WF3 data sets
Executes prefix resolution and prefix update functions
Supports IMS Parallel Reorganization, single job step execution of database
reorganization, prefix resolution, and prefix update tasks
Supports the following three modes of operation:
Stand-Alone Prefix Resolution
• Replacement of IMS Database Prefix Resolution utility (DFSURG10)
Stand-Alone Prefix Update
• Replacement of the IMS Database Prefix Update utility (DFSURGP0),
Concurrent Prefix Update
• Running prefix resolution and prefix update in a single job step
• This function is not available in IMS standard utilities
© 2015 IBM Corporation 53
IMS High Performance Image Copy
High performance utility to create database back ups easily and quickly
For both Full Function (including HALDB) and Fast Path DEDB
‒ Batch IC and Concurrent IC
‒ Standard IC and DFSMS COPY IC
Integration with reorganization utilities included in the Database Solution Pack
IMS Reorganization Expert
IMS Online Reorganization Facility
Integration with High Performance Pointer Checker to ensure integrity of the
backup taken
Can be executed as a stand alone utility to create backups necessary to support a
recovery strategy
Provides a quick copy method (FlashCopy) with simple interface
Create stacked tape easily without special JCL control
Autonomics capability: switch copy technique based on database state
© 2015 IBM Corporation 54
HPIC Quick Copy Method for Stacked Image Copies
Requirement:
Reduce DB outage time when
creating stacked image copy
Solution: HPIC
Create shadow DB using FlashCopy
DB is brought back online
automatically
Read DB records from shadow
database
Create stacked output
DBDS1
DBDS2
DBDS3
Stop HDAM1 database
Read record in DBDS1
Put record into ICDS
Read record in DBDS2
Put record into ICDS
Read record in DBDS3
Put record into ICDS STACK
HDAM1
Delete shadow
Create Shadow by FlashCopy
Start HDAM1 database
DBDS1
DBDS2
DBDS3
Shadow
IMS command
IMS command
IMS High Performance Image Copy
© 2015 IBM Corporation 55
Autonomics: Internally switches IC process to CIC
Requirement:
Daily backup needed
Batch Image Copy is ideal but
active database may be busy when
job is submitted
Solution: HPIC
Checks DB authorization status,
and switch Image Copy process
Batch Image Copy (IC) to
Concurrent Image Copy (CIC)
automatically
55
DB1 DB2
Program Program Program
Suspend DB by DB QUIESCE
DB1 Busy (Command timeout )
Stop DB2
Check DB authorization status
RECONs
CIC DB1
Request Clean IC
IC DB2
IMS High Performance Image Copy
© 2015 IBM Corporation 56
Integration with HP Pointer Checker
Pseudo Online Pointer Check
DB suspended by DB QUIESCE
Shadow DB created by FlashCopy
after DB is quiesced
Checking data in parallel
DBDS1
DBDS2
DBDS3
Suspend DB by DB QUIESCE
Read record in DBDS1
Run pointer check
Read record in DBDS2
Run pointer check
Read record in DBDS3
Run pointer check
HDAM1
Delete shadow
Create Shadow by FlashCopy
Stop DB QUIESCE
DBDS1
DBDS2
DBDS3
Shadow Pointer Evaluation
IMS High Performance Image Copy
© 2015 IBM Corporation 57
IMS High Performance Pointer Checker
Ensures IMS Full Function and HALDB databases are operational, tuned,
repaired, and ready for use.
Consists of six utilities:
HD Pointer Checker (HPPC) – Main Component
• Two modes of execution
• Standard (Full) checking
• Hash checking
• Stand alone execution or integration with High Performance Image Copy
Space Monitor
HD Tuning Aid
DB Historical Data Analyzer
DB Segment Restructure
IMS Database Repair Facility
© 2015 IBM Corporation 58
58
HPPC reports condition of your IMS databases
Database validation – reports if databases
are error free or not
Statistics analysis – generates statistical
reports and keeps data for historical reasons
DBA needs to know database status
Damaged
Faulty
operation
Software or hardware error
Disaster
Space Shortage
Growth of data
Fragmentation
Data update
……
Reorganization
Backup
Recovery
Database Statistics
Analysis
© 2015 IBM Corporation
IMS High Performance Pointer Checker
© 2015 IBM Corporation 59
© 2014 IBM 59
HD Pointer Checker
Detecting errors in a database •Finds errors by checking direct pointers (physical pointers, logical pointers, hierarchical pointers,
pointers to free space, and index pointers) in IMS Full-Function and HALDB databases.
Describing the database
Prints a detailed description of the condition of the IMS database. This includes comprehensive
statistical information about pointers, segments, and free space.
Printing HDAM, HIDAM, PHDAM and PHIDAM database blocks (BLOCKMAP)
•Prints hexadecimal and character dumps of database control intervals or blocks. Each dump
includes a map of all the segments and free space elements in the block, as well as the relative
byte address (RBA) for each printed line.
Finding absolute disk addresses
•Converts an RBA into the corresponding absolute disk address. It calculates the cylinder, track,
record, and offset for each input RBA
Finding all pointers to a target segment
•Lists the RBAs of all pointers (in other segments, possibly in other databases) that point to a
specific target segment
Creating history records using HPPC ‘HISTORY dataset’
© 2015 IBM Corporation
IMS High Performance Pointer Checker
© 2015 IBM Corporation 60
60
Space Monitor Utility
It reports space information of database data set and monitors threshold for space information.
Space Monitor can run as a stand-alone program and can run in HDPC.
HDPC analyzes statistics from the aspect of IMS data, and Space Monitor analyzes them from the aspect of physically data set. You can get both kinds of statistics when you use HDPC and SPMN together.
//HDPC JOB
// EXEC
// :
//PROCCTL DD *
OPTION SPMN=YES, THRESHOLDS=…
//SPMNSPDT DD DSN=
HDPC reports
Space Monitor reports
DB HDPC
SPMN
© 2015 IBM Corporation
IMS High Performance Pointer Checker
© 2015 IBM Corporation 61
61
Space Monitor gets data set information from catalog and VTOC, analyzes them, and generates space statistics reports. For example,
Current space information:
• Data set size, Used space, the number of extents, free space, CI/CA Splits, allocated volumes, and etc..
Estimate available space, available extents, available volumes, and etc..
You can set thresholds for the space information by PROCCTL control statement or SPMNIN control card. If both of them are specified, PROCCTL is used.
It is recommended to use PROCCTL. It is easier than SPMNIN.
The benefits of SPMNIN is it can be used for both SPMN stand alone and HDPC.
//PROCCTL DD *
PROC ….
OPTION SPMNIN=YES, THRESHOLDS=(EXTENTS=13,USEDSPC%=95) DATABASE …. /*
//SPMNIN DD *
DBNAMEX DDNAMEX SAMPDSNAME 12020 10
/*
© 2015 IBM Corporation
IMS High Performance Pointer Checker
Space Monitor Utility
© 2015 IBM Corporation 62
HD Tuning Aid
Evaluate HDAM & HIDAM randomizer settings
Assist with tuning analysis:
– Current randomizer performance in (P)HDAM databases
– Current root segment location in (P)HIDAM databases
– Potential randomizer performance in (P)HDAM databases
– Potential randomizer performance if (P)HIDAM converted to HDAM
"KEYSIN" dataset must be created to run HD Tuning Aid
– Created from HDPC scan execution
IMS High Performance Pointer Checker
Analyzes the KEYSIN data set
• KEYSIN is a unique data set of HPPC
• KEYSIN is created by HDPC. It contains root key
HDTA cannot run in HDPC. Use HPPC sample cataloged procedure FABPPTA. It contains HDPC job step and HDTA job step in one job.
DB HDPC
KEYSIN
HDTA
© 2015 IBM Corporation 63
63
Managing Historical Statistics Data
The database statistics information are stored in :
IMS Tools KB output repository Common Repository of IMS Tools
History data set Unique file of HPPC
DB
HDPC
Statistics data
ITKB Server
History data set Output Repository
Statistics are recorded historically
HDPC &SPMN Reports
SPMN
© 2015 IBM Corporation
IMS High Performance Pointer Checker
© 2015 IBM Corporation 64
64
DB Historical Data Analyzer
Data collected and reported by HP Pointer Checker can be used to create
several reports.
History Data by DB-DS Report
History Data set by Key Date Report
HD Pointer Checker Summary Report
HD Analysis Report
Typical Uses:
Summarization of historical Pointer Checker information
To provide a graphical representation in the form of graphs,charts or tables showing
dataset allocation and usage of IMS databases.
Export the database statistics data from the HISTORY dataset to a flat file that can
be processed by user application programs
© 2015 IBM Corporation
IMS High Performance Pointer Checker
© 2015 IBM Corporation 65
65
DB Segment Restructure
Can perform tasks that would otherwise be extremely difficult including:
Initializing fields with specific values.
Moving data to different positions in a segment.
Changing the DB hierarchy by:
• combining segments
• splitting segments
• deleting segments
• moving segments to new hierarchic location
Creating a test database using part of a larger database
Converting database organization type
• HDAM database to HIDAM
• HIDAM and HISAM databases to HDAM
• HSAM database to HISAM
• HDAM, HIDAM, and HISAM databases to DEDB
Typical Uses:
- Restructure DBs (unload followed by load)
- Convert DB to another type
- Create test DB
© 2015 IBM Corporation
IMS High Performance Pointer Checker
© 2015 IBM Corporation 66
66
DB Segment Restructure
Old Hierarchy
New Hierarchy
with History
segment moved
which changes
segment codes for
History, Deliver
and Schedule
© 2015 IBM Corporation
IMS High Performance Pointer Checker
© 2015 IBM Corporation 67
67
• Repairs IMS databases that contain pointer or data errors
• Interactive Mode:
– Verify data interactively in ISPF
– View data
RBA Block Key
– Navigate by following pointers
– Repair (ZAP) pointers or data
– Can undo mistakes, or save changes to a batch job
• Batch mode: – Replaces VSAM Zapper product (but handles OSAM or VSAM) – Can verify data before replacing it but no “UNDO” capability
– Pre-checking of VER and REP
• Audit trail of all database repair activity
Database Repair Facility
© 2015 IBM Corporation
IMS High Performance Pointer Checker
© 2015 IBM Corporation 68
HALDB Toolkit
Provides functions that can be used for:
Simulating conversion to HALDB
Converting to HALDB
Re-partitioning of a HALDB
Maintaining a HALDB
Analyzer function
DBRC handling
Miscellaneous utilities which assist in HALDB handling tasks
• Indirect list key (ILK) rebuild
• This utility enables you to rebuild all ILKs and their references, and allows you to resolve
conflicting ILKs.
© 2015 IBM Corporation 69
CONVERT command
Backup
HALDB Toolkit
CONVERT
DBDLIB/ACBLIB
RECON FF DB
MDALIB
DBDLIB
RECON
IC datasets
HALDB
backup
//IHCSYSIN DD *
CONVERT –
DBD(HDAM1) –
DSNPREF(HALDB.HDAM1) –
PARTNUM(3) -
:
HSSR Engine
HP Load
HPIC+HPPC Hash
© 2015 IBM Corporation 70
Convert / Migrate to HALDB
CONVERT command can be used to:
•convert to HALDB in a single step batch job
•Integrate with ORF to allow ‘online’ conversion
Partitioning can be determined by:
•PARTNUM: specify the number of partitions
•PARTSIZE: specify the size of partitions
•KEYS: specify the high keys
ISPF Interface available to help orderly tracking of large conversion projects
Database Types that can be converted:
•HDAM to PHDAM
•HIDAM to PHIDAM
•HISAM to PHIDAM
•SHISAM to PHIDAM
•PDB/PDF to PHDAM/PHIDAM
© 2015 IBM Corporation 71
Operations of DL/I Batch program for HALDB with HALDB Toolkit
• HALDB Toolkit provides a function to treat a HALDB as if it is not registered.
(DBRC=N) • No SUBSYS record created
• No batch log created
• Normal procedure:
(1) /DBR
(2) Create an image copy#1
(3) Run DL/I Batch program
(4) Create an image copy#2
(5) /STA
DL/I Batch program
(DBRC=Y)
Create IC
IC#1
/DBR DB Create IC
IC#2
/STA DB
• Recovery procedure:
(1) Re-allocate DBDS
(2) Recovery from image copy#1
(3) Re-build ILDS/Primary Index
(4) Re-run DL/I Batch program
from the beginning
RECON info. SUBSYS record
Batch log record IC record IC record
RECON
X
© 2015 IBM Corporation 72
Library Integrity Utilities
A set of utilities designed to validate, compare, map, report, and regenerate a number of
IMS libraries.
IMS libraries supported
• ACB,DBD,PSB, and MFS libraries
• RECON data sets
• IMS Catalog
LIU utilities
• DBD/PSB/ACB Reversal
• DBD/PSB/ACB Mapper
• DBD/PSB/ACB Compare
• Consistency Checker
• Multiple Resource Checker
• Advanced ACBGEN, ACBLIB Analyzer
• MFS Reversal, MFS Compare
• Integrity Checker
• Catalog Manager =
© 2015 IBM Corporation 73
Advanced ACBGEN Utility
Advanced ACBGEN is faster than IMS standard ACBGEN.
Reduces I/O, elapsed time and CPU time significantly.
In our test system, IMS ACBGEN takes 97 seconds, Advanced ACBGEN takes 14
seconds to build 2600 members.
Preview function
Prior to an ACBGEN, a user can know how many members and which members will be
built.
Getting ACBGEN result in more concise format and unique information
A user can know which members are built.
Information of processed members Before and after ACBGEN.
© 2015 IBM Corporation 74
Multiple Resource Checker
DBD Library PSB Library
ACB Library RECON
DBD Library DBD Library
PSB Library PSB Library
ACB Library ACB Library RECON
RECON
Resource Check
Summary Report
Multiple
Resource
Checker
Input: Multiple Resources (DBD, PSB, ACB and RECON)
Up to 10 DBD/PSB/ACB libraries, 10 RECONs
If a user has many resources…..
• Other compare utilities • Process 1-1 for two resources in one job step.
• Need to run many jobs and analyze many reports for many members.
• Multiple Resource Checker compares multiple IMS resource in one job step. • Prints a consolidated summary report.
© 2015 IBM Corporation 75
IMS Catalog – An issue
DBDLIB
PSBLIB
ACBLIB ACBGEN
IMS ACBLIB and IMS Catalog must be synchronized.
However, there is no method to confirm whether ACBLIB, IMS Catalog and
directory are consistent.
Are IMS Catalog and
ACBLIB
synchronized?
? Populate
Are ACBs in the catalog consistent
to ACBLIB and consistent within IMS Catalog?
DBD
PSB
IMS Catalog
Directory
ACB (DMB) ACB (PSB)
Solution: LIU Catalog Manager Utility
© 2015 IBM Corporation 76
Catalog Manager Utility
Ensures DBDs and PSBs in the IMS catalog are maintained correctly.
Time-stamp validation
• Checks time-stamp of each DBD and PSB between ACBLIB and IMS Catalog
• Checks time-stamp of each ACB (DMB and PSB) member in the directory, If IMS
loading ACB from Catalog is activated
Database Version validation
• If database versioning is enabled, the utility also checks whether the database
version numbers are the same for each DBD.
WTO message & IMS Catalog Validation report
© 2015 IBM Corporation 77
Catalog Manager Utility - WTO messages
RC=00 : All of DBDs and PSBs are consistent between ACBLIB and IMS Catalog
RC=04 : Some of DBDs and PSBs are inconsistent between ACBLIB and IMS
Catalog.
If inconsistency is found, run the ACB Generation and Catalog Populate utility
(DFS3UACB) or the IMS Catalog Population utility (DFS3PU00) to correct the
IMS catalog.
© 2015 IBM Corporation 78
IMS Catalog Validation Report
Valid : same time-stamp and DB Version between ACBLIB and IMS Catalog
Invalid : different time stamp or member does not exist in IMS catalog
This report is available
for IMS V12 or V13
© 2015 IBM Corporation 79
Summary
IMS Database Solution Pack is a single, simplified, integrated solution which includes all
the necessary components to effectively and efficiently manage your IMS Full Function
Database Environment
IMS Online Reorganization Facility
IMS High Availability Large Database Toolkit
IMS Database Reorganization Expert
IMS High Performance Unload
IMS High Performance Load
IMS High Performance Prefix Resolution
IMS Index Builder
IMS High Performance Image Copy
IMS High Performance Pointer Checker
IMS Database Repair Facility (delivered as part of HP Pointer Checker)
IMS Library Integrity Utilities
© 2015 IBM Corporation 80
IMS Tools from IBM
Providing Modernization and Autonomics
IMS Fast Path Solution Pack
Autonomics IMS HP Fast Path Utilities IMS DIB Repair Facility IMS HP Image Copy IMS Library Integrity Utilities
IMS Recovery Solution Pack
IMS HP Image Copy IMS Database Recovery Facility IMS HP Change Accumulation IMS Index Builder IMS DRF Extended Functions
IMS Performance Solution Pack IMS Connect Extensions IMS Performance Analyzer IMS Problem Investigator
IMS Database Solution Pack Autonomics IMS DB Reorganization Expert - Unload, Load, Index Build, Prefix Resolution/Update IMS HP Image Copy IMS HP Pointer Checker IMS Library Integrity Utilities IMS HALDB Toolkit IMS Online Reorg Facility
IMS Sequential Randomizer Generator Data Base Administration
IMS Cloning Tool IMS Database Control Suite
Utility Management
IMS HP Image Copy IMS DEDB Fast Recovery IMS Recovery Expert
Backup and Recovery Batch Terminal Simulator
Program Restart Facility Application Management
Performance Management
IBM Transaction Analysis Workbench IMS Buffer Pool Analyzer IMS Network Compression Facility
System / TM Administration
System IMS Command Control Facility IMS ETO Support IMS HP Sysgen Tools IMS Queue Control Facility IMS Workload Router TM IMS Configuration Manager IMS Sysplex Manager
Regulatory Compliance
Guardium ITAP for IMS IBM Infosphere Guardium Data Encryption for DB2 and IMS Databases
IBM Tools Base for z/OS
© 2015 IBM Corporation 81
Thank You
Richard Phipps IBM IMS and IMS Tools
© 2015 IBM Corporation 82
Legal Disclaimer
• © IBM Corporation 2014. All Rights Reserved.
• The information contained in this publication is provided for informational purposes only. While efforts were made to verify the completeness and accuracy of the information contained
in this publication, it is provided AS IS without warranty of any kind, express or implied. In addition, this information is based on IBM’s current product plans and strategy, which are
subject to change by IBM without notice. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this publication or any other materials. Nothing
contained in this publication is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and
conditions of the applicable license agreement governing the use of IBM software.
• References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or
capabilities referenced in this presentation may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to
future product or feature availability in any way. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by
you will result in any specific sales, revenue growth or other results.
• If the text contains performance statistics or references to benchmarks, insert the following language; otherwise delete:
Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will
experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage
configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.
• If the text includes any customer examples, please confirm we have prior written approval from such customer and insert the following language; otherwise delete:
All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs
and performance characteristics may vary by customer.
• Please review text for proper trademark attribution of IBM products. At first use, each product name must be the full name and include appropriate trademark symbols (e.g., IBM
Lotus® Sametime® Unyte™). Subsequent references can drop “IBM” but should include the proper branding (e.g., Lotus Sametime Gateway, or WebSphere Application Server).
Please refer to http://www.ibm.com/legal/copytrade.shtml for guidance on which trademarks require the ® or ™ symbol. Do not use abbreviations for IBM product names in your
presentation. All product names must be used as adjectives rather than nouns. Please list all of the trademarks that you use in your presentation as follows; delete any not included in
your presentation. IBM, the IBM logo, Lotus, Lotus Notes, Notes, Domino, Quickr, Sametime, WebSphere, UC2, PartnerWorld and Lotusphere are trademarks of International
Business Machines Corporation in the United States, other countries, or both. Unyte is a trademark of WebDialogs, Inc., in the United States, other countries, or both.
• If you reference Adobe® in the text, please mark the first use and include the following; otherwise delete:
Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other
countries.
• If you reference Java™ in the text, please mark the first use and include the following; otherwise delete:
Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both.
• If you reference Microsoft® and/or Windows® in the text, please mark the first use and include the following, as applicable; otherwise delete:
Microsoft and Windows are trademarks of Microsoft Corporation in the United States, other countries, or both.
• If you reference Intel® and/or any of the following Intel products in the text, please mark the first use and include those that you use as follows; otherwise delete:
Intel, Intel Centrino, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States
and other countries.
• If you reference UNIX® in the text, please mark the first use and include the following; otherwise delete:
UNIX is a registered trademark of The Open Group in the United States and other countries.
• If you reference Linux® in your presentation, please mark the first use and include the following; otherwise delete:
Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of
others.
• If the text/graphics include screenshots, no actual IBM employee names may be used (even your own), if your screenshots include fictitious company names (e.g., Renovations, Zeta
Bank, Acme) please update and insert the following; otherwise delete: All references to [insert fictitious company name] refer to a fictitious company and are used for illustration
purposes only.