silicon valley data tech summit · 18/09/2019 · •hands-on training with the latest ibm...

IBM & IDUG Data Tech SummitSilicon Valley Lab | October 2-4, 2019

Silicon Valley Data Tech Summit

• Over 50 sessions delivered by lab experts with 10 sessions on user experiences, the Data Tech Summit is packed with opportunities to learn from the people who know Db2 best, including:

• Hands-on training with the latest IBM offerings, Db2 AI, Watson Machine Learning, IDAA, DevOps, and more

• Technical Deep Dives on Db2 for z/OS, Application Development, Analytics, AI/ML, and Solutions & Utilities.

• Meet the Lab Reception on Thursday evening

Insights on Data, Analytics, AI, Machine Learning, and Open Source

Merv Adrian, Gartner

Data & AI - Client Panel - Making Data Ready for AI

Daniel Hernandez, IBM VP Offering Mgmt

Enterprise AI: World-Class Analytics on IBM Z

Namik Hrle, IBM Fellow

Keynote Speakers:

https://www.idug.org/p/cm/ld/fid=2050

© 2019 IBM Corporation2

Db2 for z/OS: Continuous Delivery -

How is Db2 implementing it?• MDUG – Michigan Db2 User Group

• Mark Rader, Db2 for z/OS

• IBM Washington Systems Center

• September 18, 2019


Agenda

▪Retooling the migration process

−Continuous Delivery - What is it?

▪Migrating to Db2 12 - New terminology

▪Function Levels

−ACTIVATE command

▪Application Compatibility

▪What’s next?


Continuous Delivery

Traditional Db2 release delivery process:

3+ year cycle

Too long to get new function out into the field

With continuous delivery (CD):

Code delivered directly into service stream

After rigorous testing

Might be associated with new function level

No set schedule

Always working on the next new thing – Agile!


Db2 12 for z/OS

New Terminology


New TerminologyModes are now called……

Function levels – no longer use ‘modes’

At GA time only had two possible function levels:

V12R1M100 - new function not available

V12R1M500 - new function available

Now have:

V12R1M501 – V12R1M505

Catalog and code levels – same format as function levels

At GA time only had one catalog level: V12R1M500

V12R1M502, V12R1M503 and V12R1M505 have catalog changes


New Terminology

* (i.e. star) function levels? State after subsystem/group has been to a higher

function level (e.g. V12R1M500) and ‘activated’ an earlier function level: (e.g.

V12R1M100)

V12R1M100* (had previously been to V12R1M500 or higher FL)



No longer use CM and NFM terminology


Db2 12 for z/OS

Migration process


The standard DSNTIJTC migration job is all you need!CATMAINT UPDATE LEVEL V12R1M500

Once DSNTIJTC processing completes the subsystem/group is now V12!

It is now at function level V12R1M100.

New SQL function cannot yet be used

Db2 12 migrations are a single step

ENFM


Migration overview: Db2 11 to Db2 12Db2 12

(FL V12R1M500)

Db2 12

Catalog

Data Sharing

Coexistence

Db2 12

(FL V12R1M100)

DSNTIJTC

(CATMAINT

UPDATE

LEVEL

V12R1M500)

DSNTIJAF

(ACTIVATE

FUNCTION LEVEL

(V12R1M500))

Db2 11

Catalog

Db2 12

Libraries

Db2 11

Libraries

Db2 11 New

Function Mode

(NFM) With SPE

1 – 2 months

1 week

Use APPLCOMPAT(V11R1) here

or APPLCOMPAT(V10R1)

Use APPLCOMPAT(V10 R1 or V11R1)

Or APPLCOMPAT(V12R1M500)

ENFM


Db2 12 for z/OS

Falling back and

release coexistence


Migration and Fallback paths

12

Db2 11 NFM

Db2 12FL

V12R1M100*

Db2 12 FL

V12R1M100

Db2 12FL

V12R1M5001

1 DSNTIJTC

2 DSNTIJAF or ACTIVATE command

Db2 12FL

V12R1M501

Can only migrate to Db2 12 from Db2 V11 NFM

No skip release support

Cannot fall back to Db2 11 after activating function level V12R1M500 or above

Can return to Db2 12 ‘star’ function level V12R1M100*

2 2 2

2

2


Function levels


Function LevelsV12R1Mnnn format

V12R1M100 after initial migration to V12

V12R1M500 after ACTIVATE command (or with DSNTIJAF job)

When new function is released it may or may not be associated with a new function level

V12R1M501, V12R1M502, etc.

A function level might have a prereq catalog change

We anticipate few catalog changes in relation to function level changes.

Cannot ACTIVATE a function level if prereqs have not been done.

Multiple catalog changes can be done together with one job!

All members must have a code level that understands the function level being activated

Can SKIP over function levels

As long as prereqs have been handled


How to change function levels


Function Levels

Two ways to change function levels:

ACTIVATE command

-ACTIVATE FUNCTION LEVEL (V12R1M500)

Install SYSADM/SYSOPR authorization

DSNTIJAF job

Tailored by the install process

Install SYSADM/SYSOPR authorization

Function and catalog level changes are group wide

When function/catalog level changes, it changes for all!

Can differ from individual member code levels

Any member can initiate if authorized user


Activating Function Levels

Can skip function levels if desired

Can return to an earlier function level, ‘star’ function level

Db2 11 NFM

Db2 12 FL

V12R1M100*

Db2 12 FL

V12R1M100

Db2 12FL

V12R1M500

1 DSNTIJTC

2 DSNTIJAF or ACTIVATE command

Db2 12FL

V12R1M501

Db2 12FL

V12R1M502

Db2 12 FL

V12R1M500*

Db2 12 FL

V12R1M501*

1 2 2

2

2 2

2

222


Function and Member Code Levels

Can only activate to a function level if all active members in a data sharing group are running with a level of code that understands the new function level!

Very much like fallback SPE code for release boundary

Each subsystem has it’s own code level. Same format as catalog and function levels (ie V12R1M500)

Each group member’s code level can be different

Examples follow


Function Levels

Example 1: Current group function level is V12R1M500

Member B puts on PTF maintenance that includes V12R1M501 code level

ACTIVATE FUNCTION LEVEL V12R1M501?

No – DB2A does not know anything about V12R1M501

Only DB2B understands V12R1M501

DB2A only understands V12R1M100and V12R1M500

DB2ACode level

V12R1M500

DB2BCode level

V12R1M500

DB2BCode level

V12R1M501

Function Level

V12R1M500


Function Levels

Example 2: Current group function level is V12R1M503Note: both members have code levels higher than group function level

Member B puts on PTF maintenance that includes V12R1M509 code level


No – DB2A does not know anything about V12R1M509. Only DB2B understands V12R1M509


Yes – both members have code levels greater than or equal to V12R1M505




Function Level

V12R1M503

DB2ACode level

V12R1M505

DB2BCode level

V12R1M507

DB2BCode level

V12R1M509


Function Levels

Example 3: Current group function level is V12R1M500

Note: both members have code levels higher than group function leveland there is a catalog change at V12R1M502 that has not yet been been made. Catalog is currently at initial V12 V12R1M500 level


No – Because V12R1M502 function level prereqs V12R1M502 catalog change

DB2ACode level

V12R1M502

DB2BCode level

V12R1M502

Current

catalog

V12R1M500

V12R1M502

catalog

Function level

V12R1M502Function level

V12R1M500


What is my function level?


Determining Function Levels

There are several ways to determine what your current function level is:

ACTIVATE FUNCTION LEVEL output

DISPLAY GROUP

DSNG007I message on MVS console is issued each time the catalog(DSNDB06) DBD is loaded into memory

DSNG014I message on MVS console is issued each time a catalog level or function level is changed

Built-in global variables: SYSIBM.PRODUCTID_EXT,

SYSIBM.CATALOG_LEVEL

FUNCTION_LVL columns in various catalog tables and also in DSN_STATEMNT_TABLE


Determining Function Levels

ACTIVATE FUNCTION LEVEL output

ACTIVATE FUNCTION LEVEL (V12R1M500)


Can I activate a function level?Not sure if a subsystem or group can be taken to a function level?

Use the ACTIVATE command with the TEST option


DISPLAY GROUP in V11 NFMData sharing with 2 members


DISPLAY GROUP in V11/V12 CoexistenceData sharing with 2 members


DISPLAY GROUP in V12Data sharing with 2 members


DSNG014I message

DSNG014I issued to MVS console anytime the catalog level or

function level changed:

Function level changed; initiated on Member Db2A

Catalog level changed; intiated on Member DB2B


What have I done?!


SYSLEVELUPDATES catalog table

▪ Records what has been done to the catalog, function level changes, and

maintenance


Application Compatibility

(APPLCOMPAT)


APPLCOMPAT – Application CompatibilitySeparates system migration from application migration

APPLCOMPAT zParm provides default for BIND/REBIND – static SQL

V10R1 for Db2 10 SQL behavior

V11R1 for Db2 11 SQL behavior

Default is V12R1M500 for Install, V11R1 for migration

BIND/REBIND option to override zParm default

CURRENT APPLICATION COMPATIBILITY special register – dynamic SQL

Migration automatically sets V11R1 prior to V12R1M500 … otherwise

DSNT225I -DSN BIND ERROR FOR PACKAGE location.collid.member

APPLCOMPAT(V12R1M500) OPTION IS NOT SUPPORTED

IFCID376 – Summary of V12 function usage

IFCID366 – Detail of V12 function usage, identifies packages that contain

incompatible SQL or XML statements


APPLCOMPAT – Application Compatibility

The APPLCOMPAT bind option is intended to ensure application behavior is consistent across releases of Db2

As such, APPLCOMPAT for the current Db2 12 release may not be specified until function level V12R1M500

Once in function level V12R1M500 or a later FL either APPLCOMPAT(n) - APPLCOMPAT(n-?) may be specified

In a future release (?) APPLCOMPAT(n-2) may be specified

TBD with continuous delivery


APPLCOMPAT – Application Compatibility

▪ Changes for V12

− Used to control the adoption of new SQL capabilities and enhancements of

particular function levels.

− Can use the application compatibility level of applications to control the

adoption and use of new SQL statements and changes to existing SQL

statements that are introduced in function levels.

− Applications cannot use new SQL statements or any changed syntax in

existing SQL statements until the application compatibility value is greater

than or equal to the function level that introduced the changes.

− The application compatibility level applies to all static and dynamic SQL

statements and data definition statements.


What’s new?


What’s new?

▪Function level 501 (ie Function Level V12R1M501)!

−Available on April 13, 2017

− PI70535 – UI45986,

− LISTAGG function

−No catalog changes

− Test run for CD!


What’s newer?


−Available on April 30, 2018

− PI95511 – UI55093

−Dataset Encryption – managing key labels

• Db2-specific support of IBM Pervasive Encryption

−Casting enhancements

−Catalog changes required (CATMAINT UPDATE LEVEL V12R1M502)


Db2 and data set encryption – V12R1M502▪ Encrypted data in data sets with key label specified

▪ Setting key label:

Db2 11 or Db2 12 M100+

Native z/OS options:

1. RACF data set profile

2. Explicit definition

• Active logs: IDCAMS

• Utility data sets: JCL

DSKEYLBL

• User managed objects:

IDCAMS

3. SMS data class

Db2 12 V12R1M502

• zPARM ENCRYPTION_KEYLABEL

• Encrypt new catalog/directory objects

• Encrypt new archive log data sets

• SQL ALTER/CREATE with TABLE or

STOGROUP

• New option KEY LABEL

• Display options, like:

• -DIS GROUP or REPORT utility or

-DIS LOG or -DIS ARCHIVE

• Utilities, including Online REORG, allocate the new data sets as encrypted

in Db2 11 or Db2 12


What’s even newer?


−Available on October, 2018

− PH00506 – UI58434

− Support for Db2ZAI (code level required, not function level)

− Temporal incompatible changes

• Replication

• System temporal time history rows with NULLs returned

−Catalog change – new global variable (CATMAINT UPDATE LEVEL

V12R1M503)


What’s even newer?

▪V12R1M504 – PH07672 – UI61476

−Available 3/12/19

−Deprecated objects – let’s talk

−Huffman compression

−New BIFs pass-through to IDAA, SQL syntax alternative

−No catalog change. FL V12R1M504 dependent on V12R1M503 catalog.


Deprecated objects

e.g. CREATE

TABLESPACE

issued by application with

APPLCOMPAT < V12R1M504

issued by application with

APPLCOMPAT >= V12R1M504

SEGSIZE n Segmented Table Space UTS – PBG

SEGSIZE 0 &

NUMPARTS p

Classic Partitioned Table Space UTS – PBR

SEGSIZE n &

MAXPARTITIONS p

UTS – PBG UTS – PBG

SEGSIZE n &

NUMPARTS p

UTS – PBR UTS – PBR


Deprecated objects

▪Classic partitioned and segmented table spaces cannot any longer be created

by applications (e.g. DSNTEP2) with APPLCOMPAT>=V12R1M504

−Consideration: multi-table segmented table space

−Reorg to convert it to multiple UTS PBG will come in future

−Attention: DROP of such an object requires CREATE with

APPLCOMPAT<V12R1M504 to keep existing structure

▪You can no longer do the following:

−CREATE TABLE referencing an empty classic partitioned or any

segmented/simple table space

−CREATE/ALTER TABLE … ORGANIZE BY HASH

−CREATE SYNONYM

−Use CREATE ALIAS instead‘


Huffman compression

▪Requirements

− z14

− UTS only

− DSNZPARM keyword TS_COMPRESSION_TYPE:

• FIXED_LENGTH (old style compression logic)

• HUFFMAN (aka entropy encoding)

− Table space has COMPRESS YES attribute

▪Next dictionary build (REORG, LOAD REPLACE or INSERT) generates Huffman dictionary

− Which compression logic is used, can be detected by value in header page field HPGZLD (formatted by DSN1PRNT):

• L or F indicates fixed-length dictionary

• H indicates Huffman dictionary


What’s newest?

• Function level 505 (i.e. Function Level V12R1M505)!Available 6/26/19

– PH09191 - UI63807

– Has a small catalog change:COPYID integer column added to both SYSPACKDEP and SYSPACKSTMT catalog tables

– REBIND Phase-in !

– Automatic page sampling for RUNSTATS, DECFLOAT columns in indexes, BIFS for column encryption, temporal and archive transparency for WHEN in triggers


V12R1M505 – REBIND Phase-in

• Problem statement

• DBA cannot bind packages that are in use, for example

• Cannot reoptimize

• Cannot change bind options

• Cannot switch to previous access path in case of regression

• ‘in use’ means use count is > 0

• Applies to RELEASE COMMIT until commit

• Applies to RELEASE DEALLOCATE until thread termination

• Solution: REBIND phase-in

• Existing threads use current package copy

• DBA issues REBIND; Db2 creates new package copy

• Db2 marks current copy ‘do not use’ in EDM pools

• New threads use new copy

• Old copy deleted when no longer used


REBIND Phase-in – sequence of events

• Threads executing CURRENT package copy (copy ID n)

• REBIND

• Create a next CURRENT copy of package with new copy ID

(n+1)

• Replicate old CURRENT copy (n) to PREVIOUS (copy ID 1)

and ORIGINAL (copy ID 2) if needed

• Move copy ID n to SYSPACKCOPY as phased-out copy

• New threads load and execute new CURRENT copy (n+1) once

the REBIND completes


REBIND Phase-in: example

Thread 1 – copy ID 0




REBIND – copy ID 4: CURRENT


REBIND Phase-in: COPY-ID

COPYID column in SYSIBM… Remark

0, 4, 5, … 16 SYSPACKAGE CURRENT, used in wrap-around mode

1 SYSPACKCOPY PREVIOUS

2 SYSPACKCOPY ORIGINAL

3 - reserved

0, 4, 5, … 16 SYSPACKCOPY Phased-out copies, until deleted


REBIND Phase-in

• Current copy

• 1 row in SYSPACKAGE, including COPYID column

• Copy ID generated as 0, 4, 5, 6, …16 then wrap back

• 1 = PREVIOUS, 2 = ORIGINAL, 3 is reserved

• Phased-out copies

• In SYSPACKCOPY (with COPYID other than 1, 2)

• Cleaned up on subsequent REBIND

• EXPLAIN PACKAGE COPY copy-id

• CURRENT: copy ID in SYSPACKAGE.COPYID column (0 or non-0)

• Omit COPY clause: includes CURRENT, PREVIOUS, and ORIGINAL

• Changed value in PLAN_TABLE.HINT_USED

• New copy ID field in IFCID 239 package accounting

• Increased QPAC mapping size


REBIND SWITCH Phase-in

• Threads are executing CURRENT copy ID (n)

• REBIND SWITCH:

• Copy PREVIOUS or ORIGINAL to new CURRENT copy ID

(n+1)

• Copy n becomes phased-out copy

• Replicate phased-out copy (n) into PREVIOUS and

ORIGINAL if needed

• New threads can execute new CURRENT (n+1)


REBIND Phase-in eligibility

• Threads are executing CURRENT copy ID (n)

• REBIND SWITCH:

• Copy PREVIOUS or ORIGINAL to new CURRENT copy ID

(n+1)

• Copy n becomes phased-out copy

• Replicate phased-out copy (n) into PREVIOUS and ORIGINAL

if needed

• New threads can execute new CURRENT (n+1)


FREE phased-out copy

• Phased-out copy clean up on REBIND

• Query package lock holders

• Compare oldest thread’s package allocation time to the time a

copy becomes phased-out

• DSNT500I message with new reason code 00E30307 to

show thread blocking FREE

• New IFCID 393 to indicate long running thread

RELEASE(DEALLOCATE), uncommitted thread


What’s new in publications and function level introduction?▪Documenting new function levels - V12R1M505 – PH09191 - UI63807


What’s new in publications and function level introduction?

▪ Documenting function levels in publications


What’s next?


What comes next?

▪Continuous delivery is our future

−Allows us to deliver functionality to customers more quickly

−No longer need to wait 3+ years for new function

▪Will there be a Db2 13 for z/OS?


Questions?


Continuous Delivery – additional information

YouTube

Redpiece by Chris Crone

John [email protected]

Mark [email protected]

https://www.youtube.com/watch?v=ldHOnyJXT3M

REDP5469 Exploring IBM Db2 for z/OS Continuous Delivery

mailto:[email protected]

mailto:[email protected]

https://www.youtube.com/watch?v=ldHOnyJXT3M

https://www.redbooks.ibm.com/Redbooks.nsf/RedbookAbstracts/redp5469.html?Open


Continuous Delivery - Summary

Maintenance level (ML) as today

CST/RSU recommended best practice

Catalog level (CL)

Not required for every function level; required for V12R1M502

Function level (FL)

Group-wide event; command or job

Application compatibility (APPLCOMPAT or AC)

Package BIND; default in DSNZPARM if new package

Special Register

CURRENT APPLICATION COMPATIBILITY

Required to use new SQL function: DML, DDL, DCL


▪Thank you!


Db2 Connect and distributed

clients


Db2 Connect for Db2 12

▪ Supported product levels

− Any Db2 Connect versions in service: V10.5 and V11.1

▪ Recommended product levels

− Db2 Connect V10.5 FP5 or higher is recommended

▪ Db2 Connect V11.1 functions supported by Db2 V12R1M500

• Enhanced workload balancing

• Preserve prepared statement cache on rollback

• DRDA fast load requires V11.1 FP1 or higher

▪ Db2 Connect V11 platform support

− Java 8 support (JCC) – 10.5 FP7 or higher

− Next generation OS support – MAC OS, Windows 10, Windows 2016

− Entity Framework 7.0 support - .Net Provider


Db2 connect actions required for FL > M500

▪ Required driver levels to exploit FL > M500 (APPLCOMPAT of these driver packages > M500)

− JDBC and SQLJ: versions 3.72 and 4.22, or later

− Other data server clients and drivers: V11.1 Mod. 2 FP 2

▪ Modifications on client side required

− CLI and ODBC drivers

• Add parameter to <database> or <dsn> section of db2dsdriver.cfg file

• <parameter name="clientApplCompat" value="V12R1M501"/>

− JDBC and SQLJ

• change the DB2BaseDataSource.clientApplcompat Connection or DataSource property value

− Requirement removed by PH08482 !

▪ https://www.ibm.com/support/knowledgecenter/SSEPEK_12.0.0/apsg/src/tpc/db2z_applcompatclients.html

https://www.ibm.com/support/knowledgecenter/SSEPEK_12.0.0/apsg/src/tpc/db2z_applcompatclients.html


Useful links around Db2 Connect

− End-of-Support dates

• http://www-01.ibm.com/support/docview.wss?uid=swg21168270

− License information

• http://www-03.ibm.com/software/sla/sladb.nsf/displaylis/6DCDA966A893879185257FD2004FBFC9?OpenDocument

− Db2 Connect platform details for all editions

• http://www.ibm.com/support/docview.wss?uid=swg27038051

− Db2 data server client platform details via Db2 LUW link


− Db2 data server driver downloads, platform-specific

• https://www-01.ibm.com/marketing/iwm/iwm/web/reg/pick.do?source=swg-idsdpds&lang=en_US

− Db2 Client & JDBC driver versions and downloads


− IBM Knowledge Center of Db2 & Db2 Connect V11

• http://www.ibm.com/support/knowledgecenter/SSEPGG_11.1.0/com.ibm.db2.luw.welcome.doc/doc/welcome.html

http://www-01.ibm.com/support/docview.wss?uid=swg21168270

http://www-03.ibm.com/software/sla/sladb.nsf/displaylis/6DCDA966A893879185257FD2004FBFC9?OpenDocument



https://www-01.ibm.com/marketing/iwm/iwm/web/reg/pick.do?source=swg-idsdpds&lang=en_US


silicon valley data tech summit · 18/09/2019 · •hands-on training with the latest ibm...

Documents