a nuts and bolts approach to plans and packages
TRANSCRIPT
A Nuts and Bolts Approach to Plans and Packages
Rob CraneFedEx Freight System
Session: G01
Monday, May 8, 2006 • 10:20 a.m. – 11:30 a.m.
Platform: DB2 UDB for z/OS
Page 2A Nuts and Bolts Approach to Plans and Packages Page 2IDUG North America 2006
FedEx Freight Production System Summary
CICS
PIPES
DB2 Subsystem DBP1
AFW1 (13)
BATCH
CICS
PIPES
DB2 Subsystem DBP2
AFW2 (4)
BATCH
CF03 (1)
Group Buffer Pools
AFWC (1)
CA SPOOL
JHS
CF04 (1)
Shared Com Area
Lock Structure
Emergency Failover Box
Z990 – 2084-313 Z900 – 2064-1C1
Z880 ESS
SharkDASD FARM
Virtual Tape Server
64 Drives
Automated Tape Library
32 Drives – Offsite IC
5 TB data, 5,000 mips
DSNDS Group
Page 3A Nuts and Bolts Approach to Plans and Packages Page 3IDUG North America 2006
Presentation Overview
The Big Picture - precompile, bind, bind parameters and relationship between programs, plans, packages and collections
Package execution flow, system & catalog table overview - the tables that drive the process
Recommendations for implementing and maintaining plans, collections and packages
Advanced package topics & problem resolution Security considerations & SQL statements to view and
manage your plans, collections and packages Questions?
Page 4A Nuts and Bolts Approach to Plans and Packages Page 4IDUG North America 2006
Many Benefits to Packages! Enhances application enabling Reduces bind time - only bind changed DBRMs Allows multiple versions of programs Flexible naming conventions & QUALIFIER keyword Increases bind availability Granularity in bind options Access to mirror tables, set current packageset Allows remote statically bound SQL SQLJ, stored procedures and triggers Supports built in performance fallback from rebind activity due to
database upgrade/maintenance and general rebind performance work. Free the package(s) in the high order collection. (FPF – free package
fallback)
Page 5A Nuts and Bolts Approach to Plans and Packages Page 5IDUG North America 2006
Source Code & Include LibrariesHost language code and SQL statements
Precompile
Modified SourceHost language code only, SQL commented out
Object CodeLoad module in your loadlib(s)
Plan / Package
Database Request ModuleDBRM, all SQL statements
Compile & Link DB2 Bind
executes in executes in
Application Address SpaceCICS/TSO/BATCH/IMS DB2 Address Space
thread
Conceptual Picture
Page 6A Nuts and Bolts Approach to Plans and Packages Page 6IDUG North America 2006
Understanding the Relationship
Plan(s)
Collection(s)
Package(s)
Page 7A Nuts and Bolts Approach to Plans and Packages Page 7IDUG North America 2006
Plan and Package Bind
Bind plan with PKLIST Bind package
BIND PLAN(PLAN_NAME)
PKLIST(list of collections or individual DBRMs)
DSNDB01
SYSPLAN *REMARKS
SYSPACKLIST
SYSPLANAUTH
SYSPLANSYSTEM
SCT02
DSNDB06
BIND PACKAGE(collection name)
MEMBER(DBRM aka package / program name)
DSNDB06 DSNDB01
SYSPACKAGE *
SYSPACKSTMT
SYSPACKAUTH
SYSPACKDEP
SYSPKSYSTEM
SPT01
catalog directory catalog directory
V8
Page 8A Nuts and Bolts Approach to Plans and Packages Page 8IDUG North America 2006
Precompile Steps and Activity
Syntax checks SQL statements and :host variables referenced by them, including DCLGEN variables
Translates SQL into host language calls and comments out SQL, creating a parameter list
The consistency token is introduced x’487A7C9E87E7Y68E’ Stores the VERSION value in the DBRM and modified source code Does not validate DB2 objects against the catalog tables!
(DSNDB06) NEWFUN YES / IBMREQD = ‘L’
V8
Page 9A Nuts and Bolts Approach to Plans and Packages Page 9IDUG North America 2006
Precompile – NEWFUN YES
Package / Plan
Unicode Database Request ModuleDBRM, all SQL statements in UNICODE - UTF 8 - CCSID(1208)
Bind with EXPLAIN(YES)
executes in
DB2 Address Space
V8 in New Function Mode (NFM)
PLAN_TABLE
Performance Tuning
DSN_FUNCTION_TABLE DSN_STATEMNT_TABLE
V8
OPTHINT
DSN_DETCOST_TABLE DSN_FILTER_TABLE
DSN_PGRANGE_TABLE DSN_PGROUP_TABLE
DSN_PREDICAT_TABLE DSN_PTASK_TABLE
DSN_SORT_TABLE DSN_SORTKEY_TABLE
DSN_STRUCT_TABLE
Page 10A Nuts and Bolts Approach to Plans and Packages Page 10IDUG North America 2006
Bind Steps and Processes
Translates SQL into executable instructions Validates tables, columns, etc., against the DB2 catalog tables to
verify these objects exist in the subsystem you are binding to Resolves all views, alias and synonym references to the
underlying table Optimizes SQL - Access Path Selection Validates security / authorizations Explain – externalizing the access path
Page 11A Nuts and Bolts Approach to Plans and Packages Page 11IDUG North America 2006
Bind Parameters LIBRARY(pds name) - library containing the DBRMs generated from
the precompile MEMBER(dbrm name) - package name OWNER(authid) - owner of the package, explicit rights QUALIFIER(string) - identifies what variable to use for unqualified
names in the package (tables, views, etc.) ENABLE/DISABLE(specify or *) - specify environments where the
package can execute (a must for online transactions where access is usually granted to PUBLIC)
VALIDATE(bind/run) - specify BIND DYNAMICRULES(RUN) - understand the affect of BIND, particularly in
relationship to the owner of the package and the authid’s system privileges. Having SYSADM as the owner in conjunction with DYNAMICRULES(BIND) creates a security exposure for dynamic SQL.
DBPROTCOL(DRDA) - do not use private (zIIP workload)
Page 12A Nuts and Bolts Approach to Plans and Packages Page 12IDUG North America 2006
Bind Versus Rebind
Bind - creates a new plan or package from one or more DBRMs
Rebind - replaces an existing plan or package with input only from the old plan or package found in the DB2 catalog (the source code is not referenced)
Rebinds are typically used when you think access path information needs to be updated due to statistical changes in your data. Rebind is much faster on V8, it smokes! Don’t be afraid of REBIND.
When should you rebind? Do not forget trigger packages!
V8
Page 13A Nuts and Bolts Approach to Plans and Packages Page 13IDUG North America 2006
Re-Optimization Bind and Rebind
REOPT(VARS) tells DB2 to re-optimize at execution time, and applies to both static and dynamic SQL. The default setting is NOREOPT(VARS).
Access paths are still determined at bind time, but the access path is determined again at execution time based on the values of the variables (host variables, parameter markers, special registers).
PLAN_TABLE reflects the access path determined at bind time without re-optimization. You can use IFCID 22 for access path determination after re-optimization has occurred at execution time.
Be cautious using this with static SQL. REOPT(ONCE) available for dynamic SQL.
V8
Page 14A Nuts and Bolts Approach to Plans and Packages Page 14IDUG North America 2006
ACQUIRE
Recommend using ACQUIRE(USE) on all plans. USE is the only option for packages.
Tells DB2 when to acquire the intent locks and what type of lock is needed (IS or IX).
If you think you need to specify ACQUIRE(ALLOCATE), consider using LOCK TABLE IN EXCLUSIVE MODE instead. If the application intends to update every row, denote that in the application versus the bind parameter.
Page 15A Nuts and Bolts Approach to Plans and Packages Page 15IDUG North America 2006
RELEASE(COMMIT)
The old stand-by. Most recognized, understood and used. Allows DB2 to release locks as soon as possible.
Use if your application uses lock escalation, repeatable read isolation level, or mass deleting (this is your only choice).
Prepackaged software with a large number of packages tied to one plan will typically use (COMMIT).
Exclusive (X) page locks are released at commit regardless of the release parameter.
Changes were implemented with V8 to reduce locks passed to XCFAS.
DURATION
Page 16A Nuts and Bolts Approach to Plans and Packages Page 16IDUG North America 2006
RELEASE(DEALLOCATE)
Prior to V8, data sharing systems were recommended to use RELEASE(DEALLOCATE) to reduce tablespace lock activity. V8 changes increased performance of plans and packages bound with RELEASE(COMMIT) by reducing global and false contention for pageset / partition locks.
Good for batch flows with frequent commits. Let batch work take advantage of sequential detection and list prefetch.
Use for a small percentage of your protected on-line threads (20%). Persistent threads (those that remain across commits) make this choice more attractive. Determine which on-line threads/packages should use deallocate.
Eliminates traffic to the coupling facility lock structure. Good for data sharing environments and high performance oriented applications.
Thread reuse needs to occur - MONITOR this! An increase in the EDM Pool size will be needed due to the packages being held
longer in the pool. This can create issues during performance rebinds.
V8
Page 17A Nuts and Bolts Approach to Plans and Packages Page 17IDUG North America 2006
ISOLATION - Lots of Options
Cursor Stability (CS) is the most commonly used option. Locks with the movement of the cursor; only locks qualified data and dirty data if updateable. Releases U locks when you move off the page or changes to a S or X lock. Acquires and releases Shared locks in the same fashion except when CURRENTDATA(NO) is specified, then S locks are not acquired.
Repeatable Read (RR) reads the data multiple times within the same unit of work with the exact same results. Locks entire page and keeps them locked until commit. Cursors using WITH HOLD retain one page lock for positioning after commit. Resource intensive option.
Read Stability (RS) is very similar to repeatable read with the exception of when the pages are released (similar to a combination between CS and RR). If the page contains no qualifying rows, it will release the lock when it moves off the page. If the page had qualifying rows, it is not released until commit. Data can be reread in the same unit of work with the same result (including additional rows that may now qualify).
SCOPE
Page 18A Nuts and Bolts Approach to Plans and Packages Page 18IDUG North America 2006
Try Uncommitted Read (UR)
Encourage use of ISOLATION(UR) through the statement level. Good choice in most instances, especially decision support applications. What is the likelihood of your application reading incorrect data?
You can specify at the SQL statement level and override what was used at bind time. SELECT Col1, Col2 FROM TableAAA WHERE Col1=‘value’ WITH UR;
Add to your DBA spufi/QMF statements against the catalog tables. This is especially helpful in DR testing when everyone is querying the catalog to build their recovery jobs.
Still need to issue commits for read only packages that touch objects which have online reorg requirements (to get the drain)!
Page 19A Nuts and Bolts Approach to Plans and Packages Page 19IDUG North America 2006
Default is CURRENTDATA(YES). This is resource intensive. CURRENTDATA(YES) results in lock avoidance on rows that do not qualify the search criteria. CURRENTDATA(NO) may result in lock avoidance on rows that qualified.
ISOLATION(CS) CURRENTDATA(NO) will invoke lock avoidance. If your batch jobs are committing properly, DB2 can verify that qualifying rows have been committed and will get the page without acquiring page locks. Timestamp processes ensure data integrity!
Most applications tolerate lock avoidance, meaning they do not require the data on the page to remain unchanged while the cursor is on that page.
CURRENTDATA(YES) ensures that data under read only cursors is stable. DB2 ensures data under updateable cursors is stable regardless of currentdata parameter.
Use CURRENTDATA(NO) and avoid ambiguous cursors! Specify FOR READ ONLY or FOR UPDATE OF on your cursors.
Make every effort to use CURRENTDATA(NO). See currentdata query for details on DEFERPREP (sysibm.syspackage) and
EXPREDICATE (sysibm.sysplan).
CURRENTDATA
Page 20A Nuts and Bolts Approach to Plans and Packages Page 20IDUG North America 2006
EXPLAIN (YES) Keep your explain data current and accurate. Compare explain output from one software release
to the next, looking for changes. See notes for clean up tips on the PLAN_TABLE. If you have the output of previous explains, you can use Optimization Hints to return to the
previous access path if the REBIND drastically changes your access path in a harmful manner. Try to do this at the statement level vs. dbrm level.
Only use optimization hints if you will benefit (use should be limited). Previous explain output and performance data will help you make this decision. Have you opened ETRs on your current access paths using hints?
Stats Advisor and Runstats with COLGROUP functionality to slay the overuse of opthints. Add new V8 columns, update columns to match V8 lengths. Or unload/load into new V8
PLAN_TABLE, DSN_STATEMNT_TABLE to preserve your performance data and optimization hints. Nine columns change to VARCHAR(128) and seven columns were added to increase the PLAN_TABLE to 58 columns. See notes for column details.
Preserve your explain table data for your migration to V8.V7.PLAN_TABLE, V8CM_PLAN_TABLE, V8NFM.PLAN_TABLE
Ability to EXPLAIN from statements in the dynamic statement cache once in V8 NFM.DSN_STATEMENT_CACHE_AUXDSN_STATEMENT_CACHE_TABLE
V8 supports ALIASes in the OWNER key word of bind and rebind. This is helpful for directing explain output to a common set of explain tables.
V8
Page 21A Nuts and Bolts Approach to Plans and Packages Page 21IDUG North America 2006
More on REOPT & OPTHINT
REOPT(VARS) – Useful when DB2’s estimate of qualifying
rows would benefit if the real host variable was available to the optimizer at execution
Evaluate data values at runtime Limit parts for partition scans Influence join sequence
Reoptimization is done at open cursor Static SQL – package level. Consider
isolating/consolidating your reopt statements to a few static packages.
Dynamic SQL – statement level. More granular and easier to invoke on a smaller scale.
IFCID 0022 – Did you get a better access path?
Would static SQL benefit from being dynamic and taking advantage of dynamic cache?
Sounds good! Difficult to implement and maintain. Solve the root cause.
DSNZPARM change to activate. Programmers should add QUERYNO to
their code. How are you going to name and manage
your opthints? CHAR(8) to VARCHAR(128) with V8.
After code has been bound with explain yes, need to update plan_table rows to add a OPTHINT name.
To return to the good access path previously established, you would rebind your package with OPTHINT(‘ohstring’).
Verify hint is in use! SQLcode +394, HINT_USED column of PLAN_TABLE, or query the special register: CURRENT OPTIMIZATION HINT
REOPTIMIZATION OPTIMIZATION HINTS
Page 22A Nuts and Bolts Approach to Plans and Packages Page 22IDUG North America 2006
Execution Flow
DSNDB01
SKCT & SKPT pages are read into the database buffer pool
BP0
Pages copied into EDM pool, which is in DBM1 below the 2 gig line.
DBM1
SKCT
Global Dynamic Statement Cache
EDM DBD Pool
DSNHLI Entry PointParm list (DBRM name, contoken, Section #, Statement #)
SQL Call
SKPTCT
Thread Storage
V8
2 GB bar
EDM Pool
LOB
Sort PoolRid Pool
Buffer Pool
Compression Dictionary
Castout Engine Work Area
Buffer ControlBlocks
Application Copy
Page 23A Nuts and Bolts Approach to Plans and Packages Page 23IDUG North America 2006
Simplify Your Plan Binds with Package Lists
Package List - the order in which you specify packages. The order can affect performance to a slight degree. Searching for a package involves searching the DB2 directory.
*.collection.* in PKLIST - order your collections in the PKLIST by the collections in which DB2 is most likely to find your packages first. Wildcarding everything except the collection can significantly reduce or eliminate the number of plan binds needed.
location.collection.package.version - full naming convention for a package. Overkill in most instances.
Use PKLIST strategy for ease of fall back when migrating to V8. See following slide for details.
Page 24A Nuts and Bolts Approach to Plans and Packages Page 24IDUG North America 2006
Bind a Plan Once and be DONE!
BIND(OPAYROLL)
PKLIST(*.ACCOUNTING.*, *.HUMAN_RESOURCES.*,
*.NAME_ADDRESS.*,
*.INVESTMENTS.*)
BIND(OORDERS)
PKLIST(*.INVENTORY.*,
*.NAME_ADDRESS.*,
*.ACCOUNTING.*,
*.PRODUCT_DOC.*)BIND(BBIGSPND)
PKLIST(*.ACCOUNTING.*,
*.NAME_ADDRESS.*)
COLLID (collection name) has increased from CHAR(18) to VARCHAR(128) with V8.
V8
Page 25A Nuts and Bolts Approach to Plans and Packages Page 25IDUG North America 2006
Recommendations to Consider Who performs your plan, collection, and package creation
and maintenance? Consider roles and responsibilities? How many plans should you implement? What collection names should you use? How many versions of a package should you keep? How many rows of a plan_table should you keep? Would your shop benefit from a shared collection? Have you eliminated plan based SQL? Are your PKLISTs using collection wildcarding? Are you using Set Current Package Path? Easy fallback with “performance / upgrade” collections.
V8
Page 26A Nuts and Bolts Approach to Plans and Packages Page 26IDUG North America 2006
Free Package Fallback
Bind plan with PKLIST Setup once, and be done
Bind or Rebind package Seed new release in beta collection Rebind after V8 upgrade to beta
collection Free package from “BETA” collection
for immediate fallback Handle DB upgrade rebinds Isolate new code to a select group
plan
.* *.collection.*program/package
OAR *.AR_BETA.*
*.AR_APPTRIG_BETA.*
*.AR.*
*.AR_APPTRIG_BETA.*
*.AR_BETA.*CV200407
CV000086
CV000067
*.AR_BETA.*
FB789823
FB003001
package
Bind to collid AR_BETA
package
Rebind to collid AR_BETA
Plan bound to four collections
*.collection.*
Online Accounts Receivable plan bound to its “beta” and
normal collections. Collection order is important.
Instant fallback from rebinds tied to performance or database upgrades/maintenance rebinds.
Page 27A Nuts and Bolts Approach to Plans and Packages Page 27IDUG North America 2006
Collection & Plan Naming Options
-Less Administration
-More Thread Reuse
-Less Granularity in Ownership & Security
-Greater Control
-Easier Identification of Packages Running
-More Granularity in Ownership & Security
• One per Environment (Batch, CICS)
• One per Business Area
• One per Business Area / Environment
• One per Application
• One per Application / Environment
• One per Program
Page 28A Nuts and Bolts Approach to Plans and Packages Page 28IDUG North America 2006
Package & PLAN_TABLE Clean Up
Reduce the size of DSNDB01-SKPT01 (Skeleton Package Table), as well as space associated to DSNDB06 package objects and your explain tables.
SCT02 & SPT01 typically on A002 datasets for large catalogs. Quarterly online reorg of DSNDB06/DSNDB01. Do not forget to create the
J0001/I0001 datasets for the .A002 objects you will grow into during reorg. Consider the data backup and recovery requirements for the application. If you
must use the old data for a recovery or special run, you need the old DBRM associated to that data.
Tie package clean up to application releases. Do not remove your active OPTHINT rows.
For vendor software, consider using version/release naming patterns in the plans and collections to simplify clean up.
Be careful of package free scripts / commands with no where clauses or with wildcarding!
Page 29A Nuts and Bolts Approach to Plans and Packages Page 29IDUG North America 2006
SET CURRENT PACKAGE PATH PKLIST functionality for packages executed without a plan. Allows
collection lists to be searched. No PKLIST associated to DISTSERV (for remote access via DDF
only packages are used on remote server), package path allows us to cross the server one time and resolve package collection lists at the server.
Very useful for applications that access stored procedure packages from a wide variety of business areas (schemas).
Increases stored procedure nesting flexibility. User defined functions that contain multiple programs with multiple
schemas in use. Current path can be set to current package path. Special register. Easily implement user selectable isolation level logic. Enables more flexibility with managing shared SQLJ packages.
V8 featu
reV8
Page 30A Nuts and Bolts Approach to Plans and Packages Page 30IDUG North America 2006
Which collection will a SP use to find the package?
1) Was coll_id denoted on the Create Procedure statement?
2) Was the current package path special register set?
4) Use PKLIST collections’ of calling program
NO
NO
Yes
Yes
Use SP’s catalog entry
Use collections specified by SET CURRENT PACKAGE PATH statement
3) Was the current packageset special register set?
NO Yes Use collection specified by SET CURRENT PACKAGESET statement
V8
Page 31A Nuts and Bolts Approach to Plans and Packages Page 31IDUG North America 2006
Stored Procedures
BEGIN FISHTRIP SCHEDULER
:in1 = ‘brook trout’
:in2 = ‘horse back’
:in3 = ‘mountains’
SET CURRENT PACKAGESET=‘SCHEMA1’
CALL PBESTFDT( in1, in2, in3, out4)
CHECK SQLCODE-ROUTINE
WHEN(OUT4) >= 7 DAYS
SET VACATION 2 WEEKS
WHEN(OUT4) < 7 DAYS AND >= 4
SET VACATION 1 WEEK
WHEN(OUT4) <= 3 DAYS and > 0
SET VACATION 3 DAYS
EXIT FISHTRIP SCHEDULER
PRGMFISH
WLM SPAS
SCHEMA1. PBESTFDT
RRSAF - attachment calls
Type 2 driver
DB2
DBM1 locates stored procedure, verifies caller can execute, loads SP from the loadlib, executes SP in WLM SPAS.
DSNDB06.SYSOBJ DSNDB06.SYSPAKGE WITH RETURN
EXEC
Page 32A Nuts and Bolts Approach to Plans and Packages Page 32IDUG North America 2006
Stored Procedures & PACKAGE PATH
BEGIN OUTFITTER SCHEDULER
:in1 = ‘ELK’ :in2 = ‘HORSE BACK’ :in3 = ‘MOUNTAINS’ :in4 = ‘2 HORSES’ :HUNTER = ‘JACQUE PASQUINEL’
:SP_PKLIST = “HUNT_TRIPS”, “HORSE_STOCK”, “REGULATIONS” SET CURRENT PACKAGE PATH = :SP_PKLIST
CALL HNTGUIDE ( in1, in2, in3, in4, tripID)
CHECK GET SQLCODE-ROUTINE
/*Populate HUNT_TABLE */ SELECT OUTFITTER_NAME INTO :OUTFITTER FROM FINAL TABLE (INSERT INTO HUNT_TABLE (OUTFITTER_NAME, HUNTER_NAME, TRIP_ID) VALUES (:OUTFITTER, :HUNTER, :TRIPID) CHECK GET DIAGNOSTICS-ROUTINE
/* Multirow Fetch to populate hunting party array */ DECLARE HUNT_PARTY CURSOR WITH ROWSET POSITIONING FOR SELECT HUNTER, HUNTER_DOB, HUNTER_EXPERIENCE FROM HUNT_TABLE WHERE TRIPID = :TRIPID;
FETCH NEXT ROWSET FROM HUNT_PARTY FOR 10 ROWS INTO :ARRARY_HUNTER, :ARRARY_DOB, :ARRAY_EXPERIENCE; CHECK SQLCODE-ROUTINE
EXIT OUTFITTER SCHEDULER…
PRGMHUNT
HUNT_TRIPS.HNTGUIDE
WHEN(:in2) = HORSE BACK CALL HORSE (in4,horse_resID) SET :TRIP_ID = :horse_resID WHEN(:in2) = ATV CALL REGSATV(in2, atv_resID) SET :TRIP_ID = :atv_resID
WITH RETURN
HORSE_STOCK.HORSE
Reserve :in4 HORSES UPDATE HORSE_TABLE SET :horse_resID = ‘HH-E-M-001’WITH RETURN
REGULATIONS.REGSATV
Validate :in2 SET :atv_resID = ‘not valid’WITH RETURN
WLM SPAS
OUTFITTER_NAME defined as NOT NULL WITH DEFAULT “SELF GUIDED HUNT”
V8
Page 33A Nuts and Bolts Approach to Plans and Packages Page 33IDUG North America 2006
Source Code - .sqlj
SQLJ Translator
Package
SQLJ Conceptual Picture
Serialized Profile(s) - .ser
Java Class File(s) - .class
db2sqljcustomize
Modified Source - .java
Java Interpreter
contoken generated & encapsulated in .ser
customize options (-collection XX –pkgversion AUTO)
-bindoptions Package Bind
online check, -storebindoptions YES
Customized Serialized Profile - .ser
.class
.ser
.class
.ser
type 4 driver
type 4 driver
prepare / execute
Dynamic Execution Static Execution
contoken match
Page 34A Nuts and Bolts Approach to Plans and Packages Page 34IDUG North America 2006
db2sqljcustomize
Online Check Syntax & semantic validity Check HV data types to target DB data types -qualifier PROD Don’t hard code table/view owners!
-collection FO_DOCK Default collection is NULLID, recommend avoiding the
use of NULLID for SQLJ application based packages. Treat them like you would your normal packages.
Recommend upper case collection names -pkgversion AUTO
Support multiple versions of .sqlj program Inverted timestamp (ascii collating sequence) See query in notes section on the following slide
-singlepkgname HELOSQLJ Traditional 8 character upper case package name FREE vs. DROP for mixed case package names
-storebindoptions YES Ensures binds to other environments are consistent
based on bindoptions stored in the customized .ser -longpkgname
Full circle (mixed case, map to .class name better)
QUALIFIER(PROD) OWNER(FXFP)
Value specified needs authority to perform the SQL in the SQLJ program.
Person/Process doing the bind will need packadm on the collection (FO_DOCK) which will contain the package being bound.
Binder will also need bindagent to the OWNER ID V8 allows ALIAS for explain table resolution. Very helpful
for binding with an owner different from the explain tables you want populated.
EXPLAIN(YES) VALIDATE(BIND) ISOLATION(UR/CS) CURRENTDATA(NO) RELEASE(COMMIT) PROTOCOL(DRDA)
Set in DSNZPARM (DSN6SYSP DBPROTCL=DRDA) JCC Type 4 driver, DRDA connections z9 and zIPP
REOPT(NONE)
Customize Options -bindoptions
Page 35A Nuts and Bolts Approach to Plans and Packages Page 35IDUG North America 2006
SQLJ is Your Friend Static security model
User of application does not need authority on the tables/views. Helps guard the data from access outside the application layer.
Static performance model Eliminate access path changes at runtime No prepare or fiddling with dynamic statement cache It’s a package
3 R’s (reorg, runstats, rebind) Accounting detail for reliable metrics and workload planning Versioning, built in fallback with known performance characteristics
SQL is the gold standard for RDBMS access layer. I am not a fan of encapsulating/hiding SQL inside generic java “access layers”.
Portable, write once and deploy to multiple applications and RDBMS. Less code to write, store, maintain.
It’s easy enough for someone to write poorly performing SQL. Why open Pandora's box?
*SQLJ scripts for REBIND and COPY package are found in the reference section of this presentation.
Page 36A Nuts and Bolts Approach to Plans and Packages Page 36IDUG North America 2006
Triggers are Packages
Triggers based on events that can occur to a table; insert, update, delete. Not fired for utilities.
Triggers can take action BEFORE or AFTER that event. Triggers could invoke a UDF or SP that may take action outside of DB2. Before triggers typically used to validate data. After triggers ensure business rules are met. Salary cannot be updated
more than 20% unless the job role is DBA. Ensure your action text is limited with the appropriate WHERE clause.
The “firing” predicates do not limit the action scope. Do not forget REBINDs of your trigger packages. (REBIND TRIGGER PACKAGE)
Page 37A Nuts and Bolts Approach to Plans and Packages Page 37IDUG North America 2006
Fun Stuff & Problem Resolution
Obsolete packages, tactics to prune syspackage. What is valid versus operative? Consistency token problems: package(-805), plan(-818). Unavailable resources (-904); can be on many object types,
tablespaces and indexspaces tied to catalog tables, EDM pool holding onto a persistent thread/package, not using the version parameter of precompile.
Authorization errors (-922, -551, -552, -553) Execution errors (-206, -219, -204, -117) Deadlock (-911, -913)
Page 38A Nuts and Bolts Approach to Plans and Packages Page 38IDUG North America 2006
Deprecate Obsolete Packages You have identified several packages no longer needed by the business. How can you safely
remove these packages while having fallback that does not require code to be moved into production?
Verify your list of obsolete packages against your accounting data to ensure the package is not active during the previous quarter.
REBIND DISABLE(BATCH,CICS,DB2CALL,DLIBATCH,IMSBMP,IMSMPP,REMOTE,RRSAF) SYSPKSYSTEM, SYSPLSYSTEM
If an application receives a SQLCODE -807, simply rebind the package enabling the environment it runs in. REBIND ENABLE(CICS) – off to the races. SQLCODE -923 for plan based DBRMs.
DSNT408I SQLCODE = -807, ERROR: ACCESS DENIED: PACKAGE FMR7448 IS NOT ENABLED FOR ACCESS FROM BATCH DSNT418I SQLSTATE = 23509 SQLSTATE RETURN CODE
DSNT408I SQLCODE = -923, ERROR: CONNECTION NOT ESTABLISHED: DB2 ACCESS, REASON 00E3001B, TYPE 00000800, NAME FMR7448 DSNT418I SQLSTATE = 57015 SQLSTATE RETURN CODE
Update your package dependency rebind queries to exclude packages which are not enabled for any environment by adding a subselect not exists to SYSPKSYSTEM.
Free obsolete packages after they have been disabled for xx days. Package Invalidator enhancement request (MR102805442).
Page 39A Nuts and Bolts Approach to Plans and Packages Page 39IDUG North America 2006
Valid Versus Operative
The following occurs when a table is dropped that a package depends on: Prior to the drop, the package is both valid and operative.
VALID=Y, OPERATIVE=Y After the drop, the package is invalid and operative.
VALID=N, OPERATIVE=Y Using autobind (dsnzparm) at next execution, the package is marked invalid and
inoperative. VALID=N, OPERATIVE=N. Also applies to an explicit bind. The table has to be created or the source code must be changed to correct the
problem. BINDs (not REBINDs) will pick up the source code changes. Online schema changes will invalidate packages and flush the dynamic SQL statement
cache for the altered object(s). Package invalidator command (Rob’s wish, see notes for details).
V8
Page 40A Nuts and Bolts Approach to Plans and Packages Page 40IDUG North America 2006
Contoken Query
SELECT COLLID AS COLLECTION, NAME AS PACKAGE,
HEX(CONTOKEN) AS DBRM_TOKEN,
SUBSTRING(HEX(CONTOKEN),9,8) ||
SUBSTRING(HEX(CONTOKEN),1,8) AS LOADLIB_TOKEN,
VERSION, PDSNAME
FROM SYSIBM.SYSPACKAGE
WHERE A.NAME = ‘????????’
ORDER BY NAME, COLLID, VERSION WITH UR;
-------------------------------------------OUTPUT-------------------------------------------------
COLLECTION PACKAGE DBRM_TOKEN LOADLIB_TOKEN
AUTO_OLTP P6980002 15D1906116E27DD8 16E27DD815D19061
AUTO_OLTP P6980002 15D1906116EF8EC8 16EF8EC815D19061
Page 41A Nuts and Bolts Approach to Plans and Packages Page 41IDUG North America 2006
Reference Material DB2 UDB for z/OS Release Planning Guide DB2 UDB for z/OS Application Programming and SQL Guide DB2 UDB for z/OS SQL Reference DB2 Listserv (info on lister at www.idug.org) SP Redbook (www.ibm.com/redbooks) DB2 Packages: Implementation and Use - GG24-4001-00 SQLJ Development and Deployment V8 changes to Catalog & Directory tables for packages Packageset example (for use on V2.3 - V7) Queries & Scripts to help you (see slides that follow)
Packages to monitor and correct. Example of automating the script into a batch job SQLJ Scripts, Currentdata script
Security issues (see slides that follow) What security is required (use the “least privilege” approach to protect your data) Understand what is needed versus what is easiest!
Read only package implementation for developer access in production
Page 42A Nuts and Bolts Approach to Plans and Packages Page 42IDUG North America 2006
Catalog and System Tables
Database Services Component
DB2 Catalog (DSNDB06) DB2 Directory (DSNDB01)
Internal Table Information
Backup Information
DB2 Object Definitions
Plan Information
Package Information
Authorization Permissions
RI Relationships
Stored Procedure Information
Database Information (DBD01)
Log Information (SYSLGNRX)
Plan Information (SCT02)
Package Information (SPT01)
Utility Information (SYSUTILX)
See Appendix F – DB2 Catalog Tables of the SQL Reference for details on catalog table changes associated to V8.
V8
TS TB IX
20 82 119
22 85 138
V7
DSNDB06
Page 43A Nuts and Bolts Approach to Plans and Packages Page 43IDUG North America 2006
SET CURRENT PACKAGESET
Consider using this option to limit the package search process to a specific collection.
Can be an effective way to handle testing sets and verification of new application releases.
Production applications that implement mirror tables. If you bound your plan with PKLIST (*.package), it is required. This
also implies that run authorization is used since the exact package is not known at bind time.
Use from within the SP package to set collection used. Limitation, only one collection can be specified.
Page 44A Nuts and Bolts Approach to Plans and Packages Page 44IDUG North America 2006
Automate your SQL Scripts
//*-- BUILD THE REBIND FOR THE DISABLE //BLD1RBND EXEC PGM=IKJEFT01,DYNAMNBR=100, // REGION=4M //SYSTSPRT DD SYSOUT=* //SYSTSIN DD * DSN SYSTEM(DSN) RUN PROGRAM(DSNTEP2) PLAN(DSNTEP2) PARMS('/ALIGN(LHS)') - LIB('SYS2.DB2.PROD.RUNLIB.LOAD') //SYSIN DD DSN=AFDB.OBSOLETE.PGMS(DISABLE1),DISP=SHR //SYSPRINT DD DSN=AFDB.OBSOLETE.PGMS.DISABLE1.SPUFIOUT, // DISP=(NEW,CATLG,CATLG),UNIT=SYSDA,SPACE=(CYL,(10,5),RLSE) //SYSUDUMP DD SYSOUT=* //*-- SORT OUT ALL BUT REBIND FROM DSNTEP2 OUTPUT //SRT1RBND EXEC PGM=SORT,REGION=8M //SYSOUT DD SYSOUT=* //SORTIN DD DSN=AFDB.OBSOLETE.PGMS.DISABLE1.SPUFIOUT,DISP=SHR //SORTOUT DD DSN=AFDB.OBSOLETE.PGMS.DISABLE1, // DISP=(NEW,CATLG,CATLG),UNIT=SYSDA,SPACE=(CYL,(10,50),RLSE) //SYSIN DD * SORT FIELDS=COPY OPTION VLSHRT OUTFIL OUTREC=(11,80,TRAN=ALTSEQ),VTOF,VLFILL=X'40' ALTSEQ CODE=(4F40) INCLUDE COND=(12,12,CH,EQ,C'REBIND PACKA',OR, 12,12,CH,EQ,C'REBIND PLAN(',OR, 12,12,CH,EQ,C'DISABLE(BATC',OR, 12,12,CH,EQ,C'EXPLAIN(YES)') /*
Page 45A Nuts and Bolts Approach to Plans and Packages Page 45IDUG North America 2006
SQLJ COPY Package Script//SQLJCOPY JOB X,SQLJCPY,CLASS=8,MSGCLASS=X,NOTIFY=&SYSUID //BIND EXEC PGM=IKJEFT01,DYNAMNBR=20,REGION=8M //STEPLIB DD DSN=SYS2.DB2.TEST.SDSNLOAD,DISP=SHR //SYSTSPRT DD SYSOUT=* //SYSPRINT DD SYSOUT=* //SYSUDUMP DD SYSOUT=* //SYSTSIN DD * DSN SYSTEM(DSNB) BIND PACKAGE(AFDSNP.HR_HRIS) - COPY(HR_HRIS.BENDEP) COPYVER(pAHrQGBs) - ACTION(REPLACE) ISO(UR) OWNER(AFW) QUAL(PROD) BIND PACKAGE(AFDSNP.HR_HRIS) - COPY(HR_HRIS.BENDTL) COPYVER(LBOrQGBs) - ACTION(REPLACE) ISO(UR) OWNER(AFW) QUAL(PROD) BIND PACKAGE(AFDSNP.HR_HRIS) - COPY(HR_HRIS.BENENR) COPYVER(SBArQGBs) - ACTION(REPLACE) ISO(UR) OWNER(AFW) QUAL(PROD)
Page 46A Nuts and Bolts Approach to Plans and Packages Page 46IDUG North America 2006
SQLJ REBIND Package Script
//SQLJRBND JOB X,CRANE,CLASS=P,MSGCLASS=X,NOTIFY=&SYSUID //BIND EXEC PGM=IKJEFT01,DYNAMNBR=20,REGION=8M //STEPLIB DD DSN=SYS2.DB2.PROD.SDSNLOAD,DISP=SHR //SYSTSPRT DD SYSOUT=* //SYSPRINT DD SYSOUT=* //SYSUDUMP DD SYSOUT=* //SYSTSIN DD * DSN SYSTEM(DSN) REBIND PACKAGE(HR_HRIS.BENDEP.(pAHrQGBs)) - EXPLAIN(YES) OWNER(AFW) QUALIFIER(PROD) RETAIN REBIND PACKAGE(HR_HRIS.BENDTL.(LBOrQGBs)) - EXPLAIN(YES) OWNER(AFW) QUALIFIER(PROD) RETAIN REBIND PACKAGE(HR_HRIS.BENENR.(SBArQGBs)) - EXPLAIN(YES) OWNER(AFW) QUALIFIER(PROD) RETAIN
Page 47A Nuts and Bolts Approach to Plans and Packages Page 47IDUG North America 2006
Currentdata Query
SELECT COUNT(*), 'C-CURRENTDATA-YES-AMBIGUOUS-CURSOR' FROM SYSIBM.SYSPACKAGE WHERE DEFERPREP IN ('C')
UNION ALL
SELECT COUNT(*), 'PRE-CURRENTDATA' FROM SYSIBM.SYSPACKAGE WHERE DEFERPREP IN (' ')
UNION ALL
SELECT COUNT(*), 'B-CURRENTDATA-NO-AMBIGUOUS-CURSOR' FROM SYSIBM.SYSPACKAGE WHERE DEFERPREP IN ('B')
UNION ALL
SELECT COUNT(*), 'A-CURRENTDATA-YES-ALL-CURSOR' FROM SYSIBM.SYSPACKAGE WHERE DEFERPREP IN ('A')
WITH UR;
Page 48A Nuts and Bolts Approach to Plans and Packages Page 48IDUG North America 2006
Packages to Monitor / Correct
Packages using RR or RS for isolation level. Packages using CURRENTDATA(YES). Packages using DYNAMICRULES(BIND) with OWNER(SYSADM). Packages deferring Validity Checking until run time. Packages using REOPT(VARS). Packages that are Invalid (require BIND/REBIND). Packages that are Inoperative (require explicit BIND/REBIND). Packages bound with EXPLAIN(NO). Packages bound with DEGREE(ANY) if not wanting parallelism
overhead.
Page 49A Nuts and Bolts Approach to Plans and Packages Page 49IDUG North America 2006
Queries to View the Catalog
View all the packages dependent on V8 of DB2 UDB for z/OS. (1)
View all packages for a given collection and look at the validity of the packages. (2)
View users’ privileges held on a package. (3)
Use WITH UR on all catalog queries!
Page 50A Nuts and Bolts Approach to Plans and Packages Page 50IDUG North America 2006
Are you using only packaged based DBRMs?
Determine which plans are not using collections. Look for AVGSIZE = 0 means package use only. This is what you want. (4)
Find all packages tied to a plan. (5)
Find all collections tied to a package. (6)
Page 51A Nuts and Bolts Approach to Plans and Packages Page 51IDUG North America 2006
Plan & Package Dependency Queries
What packages will need to be rebound if the the SHIPMENT and SHIPMENT_ITEM table have an online schema evolution column alters performed? (7)
Which programs Insert, Update, Delete rows from the
SHIPMENT_ITEM table? (8)
Page 52A Nuts and Bolts Approach to Plans and Packages Page 52IDUG North America 2006
Package Security
The execute privilege allows the authid with that privilege to include the package in the PKLIST of the plan bind.
Packages are normally not accessed without a plan (prior to SQLJ). Triggers, Stored Procedures and DRDA connections (SQLJ packages) can access packages directly. For these packages DB2 checks the execute privilege at run time for them not the plan.
For traditional batch and CICS programs the plan is what is run. The execute privilege on the plan denotes which authid can run the plan, with the SQL found inside the package.
Package operations (Bind Add, Bind Replace, Rebind, Copy, Free, Drop, Execute, Grant All)
Page 53A Nuts and Bolts Approach to Plans and Packages Page 53IDUG North America 2006
Package Privileges - Who & What
Based on DB2 install parameter. Either BINDADD or BIND. BINDADD is more restrictive and is required by the user to add(bind) a package and new versions. BIND is required by the user to add new versions of a package, not to create the first occurrence of a package. DSNZPARM BINDNV.
BIND, REBIND, COPY, EXECUTE granted to authid’s. Either primary or secondary. Use secondary.
DROP - the owner of the package and SYSCTRL. DROP can be useful for freeing mixed case and lower case SQLJ packages.
DROP PACKAGE “NULLID”.”BenefitEnrollements” FREE - the owner of the package, PACKADM and SYSCTRL.
Page 54A Nuts and Bolts Approach to Plans and Packages Page 54IDUG North America 2006
Collection Privileges PACKADM - Easy way to give all package privileges for each package
tied to a collection to an authid. Also gives the CREATE IN privilege. The user gets privileges for the specified collection(s).
CREATE IN - Allows you to bind a package into the specified collection.
Granting at the collection level is an effective way to simplify the tasks associated with adding new packages. Grant at the collection level versus individual packages. Execute on collection.* .
Page 55A Nuts and Bolts Approach to Plans and Packages Page 55IDUG North America 2006
Plan & Package Privileges
Who can bind a new plan or package?SYSADM, SYSCTRL authorityBINDADD authority granted to your authidCREATE IN also needed with BINDADD for packages
Who can REBIND or do a BIND(REPLACE)? SYSADM, SYSCTRL authorityBINDAGENT granted by the plan/package ownerPACKADM authority on the collection for packages or BIND
privilege for plans Who can Drop or Free a plan / package?
Page 56A Nuts and Bolts Approach to Plans and Packages Page 56IDUG North America 2006
Plan Specific Privileges
What is needed to execute a Plan?SYSADMOwnership of the planExecute privilege granted to your authid for the plan
Who can include packages in the PKLIST?SYSADM, PACKADM on the collectionOwnership of the packageExecute privilege granted to your authid for the package
Page 57A Nuts and Bolts Approach to Plans and Packages Page 57IDUG North America 2006
Read-Only Objectives
Provide a mechanism for developers to propagate read-only programs into production without impact to production systems, processes and data.
No outages to production transactions and batch processes as a result of the read-only programs. Program access must not conflict or cause production programs to
abend due to locking conflicts. Offer a solution that is segregated from production plans, collections,
packages, load libraries and DBRM libraries. Easily identify the read-only access to ensure impact to business is
acceptable. Cancel the read-only processes that don’t achieve that goal. RO SQL must be tuned, there are better ways to flush a buffer.
Not intended for business reporting or deliverables. Not for data zapping.
Page 58A Nuts and Bolts Approach to Plans and Packages Page 58IDUG North America 2006
Read-Only Developer Programs Ability for developers to execute read-only programs against production data using
DB2 packages. Developers will be the owner of the package. As the owner, explicit privileges are
inherited (bind, execute, free and the granting of those authorities to others). Developers will need:
PACKADM on their groups collection(s). Execute on their groups plan(s). Explain tables in production (or an ALIAS for their TSOID to the set you want populated). SELECT authority for objects the DBRM (SQL from your program) touches.
Developers will perform the following tasks: Program compile, precompile and package binds with EXPLAIN(YES). Follow read-only guidelines. Collaborate with DBA team to establish necessary plan/collection naming standards for
developer read-only programs. DBAs will perform the following tasks:
Plan bind to appropriate collections, with OWNER(READONLY). Creation of explain tables and indexes (or alias creation to correct explain tables). Granting developers PACKADM to appropriate collection(s). Granting of EXECUTE on the read-only plan to the developers. Granting of EXECUTE on the read-only collection to READONLY. Zparm BINDNV set to BIND. Creation of required “PROD” aliases.
Page 59A Nuts and Bolts Approach to Plans and Packages Page 59IDUG North America 2006
Read-Only Plan and Package Bind
Bind plan with PKLIST Bind package
BIND PLAN(PLAN_NAME)
PKLIST(list of collections or individual DBRMs)
ROBB####
BIND PACKAGE(collection name)
MEMBER(DBRM aka package / program name)
plan
*.RO_BB_##########.*
.* *.collection.*
*.RO_BB_##########.* P#######
program/package
ROARCV *.RO_AR_CV.*
ROFO *.RO_FO.*
*.RO_AR_FBP.*
*.RO_AR.*
*.RO_HR.*
*.RO_AR_CV.*CV200407
CV000086
CV000067
*.RO_AR_FBP.*
FB725823
FB003001
packages
bound to a collection
packages
bound to a collection
Plan bound to four collections
Plan bound to one common / shared collection
*.collection.*
60
Rob CraneFedEx Freight System
[email protected]@adelphia.net
Session: G01A Nuts and Bolts Approach to Plans and Packages
Questions?Thanks for coming!!!