DB2 12 for z/OS: DRDA Enhancements

Mark RaderIBM

Session Code: A715 November 2016 15:40-16:40| Platform: DB2 for z/OS

Objectives

• Understand how Session Tokens provide improved scalability and security for DB2 clients

• Review DB2 support for Global Transactions (XA) and learn how DB2 12 removes previous restrictions for clients deploying global transactions

• Provide overview and examples for DRDA Fast Load

22

Agenda

• Improve scalability and security for DB2 clients using Sysplex Workload Balancing (WLB)• Clients, sessions and Sysplex WLB

• Externals for DB2 12: Session Tokens

• Remove Parallel Sysplex restrictions for DB2 client global transactions• Overview of XA and current support

• DB2 12 for z/OS and XA support

• New client behavior

• Externals for DB2 12

• DRDA Fast Load

• IBM Data Server Driver properties

3

Client Challenges in a DB2 Data Sharing Environment

• Currently distributed clients must manage special register and global variable settings in a DB2 data sharing environment when different transports support a connection• Transaction pooling

• Connection reroute

• Challenges:• Special registers proliferating

• Global variables increasing

• Global variables and special registers may be returned per statement

• Global array variables in DB2 12 are significantly larger

• Global variables on client considered a security exposure

• Client and server can set these values independently

4

Sysplex WLB Session Information

• Clients using Sysplex workload balancing (WLB) must maintain session information to support transaction pooling or connection reroute• Global variables

• Special register settings

DB2A

DB2B

z/OS

z/OSClient app:

global variable1

global variable2

…

special register1

special register2

… Driver

Connection

Transport

Transport

5

DB2 12 for z/OS: Sysplex Session Token Support

• Session token• Represents current application’s session environment in DB2

• Connection attribute

• One and only one application connection

• Can span multiple transports exploiting transaction level pooling

• Only one active transport at any given time

• If session token support is not used, behavior is unchanged

• If session token support is exploited, session data will not be returned to client from DB2 for z/OS server• Security benefit

6

DB2 12: Session Token Details

• Available after Activate Function Level V12R1M500• Sysplex WLB enabled – or - Client Affinities enabled

• and clientApplcompat set to V12R1M500

• New client driver level required for CLI and Java drivers

• DB2 uses catalog tables to manage session data• SYSIBM.SYSSESSION

• SYSIBM.SYSSESSION_EX

• SYSIBM.SYSSESSION_STATUS

• Session token indexes into row containing connection session data

7

DB2 12 Session Token Support

• Client using Sysplex WLB passes session token to DB2

• DB2 maintains session data for the data sharing group• Global variables

• Special register values

• Example, if client connection rerouted to DB2B, client will pass session token on establishing transport• Global variables, special registers intact

TOKEN SESSION DATA

Token1 Global variable, spec. reg.

Token2 Global variables

Token3 Special registers

SYSIBM.SYSSESSION

DB2A

DB2B

z/OS

z/OSClient app:

Token1

Driver

Connection

Transport

Transport

8

DB2 12: Session Token - Externals

• Well-behaved applications should explicitly close result sets, statements and connections• Release database resources in a timely manner

• When application closes a logical connection• Driver flows a terminate connection to DB2

• DBAT pooled, connection inactivated, session info released

9

DB2 12: Session Token - Externals

• -MODIFY DDF command to set session data timeout value• Session data timeout: How long data will stay active before it can be

removed from SYSIBM.SYSSESSION table

• Example: -MODIFY DDF SESSIDLE(100) sets the SESSIDLE value to 100 minutes

• If session data has been purged, client will get DRDA error reply

• -30062 - Invalid session token

• Applications that receive -30062 should explicitly close the logical connection

10

Global Transactions

11

Global Transactions: Overview

• XA refers to Transaction Architecture• Programming model for coordinating changes to distributed resources

within a unit of work

• A global transaction is a unit of work that involves operations on multiple resource managers, such as DB2• All of the operations that comprise a global transaction are managed by

a transaction manager

• Example: WebSphere Application Server

12

Global Transactions: Overview

• Global transaction may have multiple branches• Transaction branches identified by a common transaction ID (XID)

• XA has a semantic for coordinating branches and sharing resources• Such as: beginning transactions, committing changes, recovery in cases

of failure

• DB2 11 and prior participated in XA with some restrictions• Did not implement entire XA semantic

13

DB2 for z/OS and Global Transactions (XA)

• XID identifies the different branches of a global transaction• Correlates different threads or processes

• XID included in Display Thread Active or Display Thread Indoubt

• XID used to correlate a transaction associated with LUWID

• Global Transaction support provided for • JDBC Type 4 driver

• For example: WebSphere Application Server

• .NET driver

• ODBC/CLI driver

• DB2 data sharing support limited• Different branches connected to different members might experience

timeouts when attempting to share resources, including locks

• DB2 Connect Server is used to direct all branches of a global transaction to one member

14

App: App:

XA

trans-

action 3

App: App:

XA

trns-

action 2

XA Transactions Accessing DB2 11 for z/OS

• Applications deploying dual transport or multi-transport model must still use DB2 Connect Server to access DB2 data sharing group• Even if applications specify Sysplex WLB, there can be no sharing

between DB2A and DB2B for different branches of a global transaction

• DB2 Connect Server is the circumvention to ensure all branches end up on one member

DB2 Connect

Server

DB2A

DB2B

z/OS

z/OSApp:

XA

trans-

action 1SCA

*

* XIDs for indoubt

1, 3

2

15

DB2 for z/OS Data Sharing and XA Support

• Indoubt list maintained in SCA in CF• Member IP address and port

• Entry added when prepared (indoubt)

• Removed after commit or rollback

• Includes XID

• Indoubt list recovered during group restart

• xa_recover returns list of prepared transactions• Includes DRDA extension

• IP:PORT for data sharing member owning the indoubt transaction

• Used by driver to send final commit decision after a failure

16

DB2 12 for z/OS and XA: New Behavior

• Sharing resources when WLB enabled• Remove need for client ‘gateway’ when Sysplex WLB is enabled

• DB2 for z/OS server can directly participate in complex transactions with multiple related connections

• Clean transaction• Extended support for Global Transaction (XA) semantics

• Dual and multi-transport transaction model

• Available in DB2 12 after Activate Function Level V12R1M500

• New client property to control new behavior• Due to behavior change (resources dropped at transaction boundaries)

• Old behavior must be supported for existing applications

• New DB2 driver level required 17

DB2 12 for z/OS: XA and Global Transaction Support

• DB2 12 supports global transactions for directly connected clients• Global transaction, begun over Transport A, tied to one DB2 member

from xa_start to xa_end

• Transport B, possibly from another application process, may receive other xa calls

• xa_prepare(readonly)

• xa_prepare plus xa_commit or xa_rollback

• xa_rollback

• If Transport B connects to different member, DB2 routes to owning member

• Examples: IBM WebSphere MQ, IBM Domino

18

DB2 12 for z/OS: XA and Global Transaction Support

• If client enables the new XA support• Seamless failover is supported and

• Seamless failover is on by default

• Session Token support required for new XA behavior

19

DB2 12 for z/OS: XA and Clean Transactions

• Prior to DB2 12, DB2 does not clean resources across transactions

• New XA support will implement clean connection reuse• Connections are reset to defaults after XA_END

• Held cursors are closed

• DGTTs are dropped

• Registers are reset to the default values

• Global variables are reset to null or their default values

20

XA Transactions and DB2 12 with Sysplex WLB

• With Sysplex WLB enabled, applications deploying XA dual-transport or multi-transport model can access DB2 12 for z/OS data sharing members after Activate Function Level V12R1M500• clientApplcompat=V12R1M500 required

• DB2 Connect Server no longer required• DB2 Connect license still required, of course

XA multi-

transport

model

DB2A

DB2B

z/OS

z/OS

App:

XA dual

transport

modelSCA

*

* XIDs: owning DB2

member + indoubts

21

DB2 12 for z/OS: XA and Sharing Resources

• Supports both loosely and tightly coupled transaction models

• All global transactions are serialized within the data sharing group

• A global transaction is owned by one member• SCA relates XID to owning member

• First member owns the global transaction• DB2 seamlessly reroutes other transactions that are part of the same

global transaction to the owning member

22

DB2 12 for z/OS: XA and Sharing Resources

• New DIS THD status characters• RX (Re-routed Xa)

• Global transaction (XA) routed from this member to owning member

• DX (Disconnected Xa)

• Displayed on owning member

• DBAT for global transaction pooled, but not available for other connections

• XA decision sequence pending

23

DB2 12 and XA Support: Example

• First connection, to DB2A, initiates global (XA) transaction• XID=00020005 00000003 00000003 336539336539

• DB2A is owning member

• Token 10

• Second connection, to DB2D, is part of same global transaction• Same XID

• Token 4 on DB2D

• Second connection routed to DB2A• Token 11 on DB2A

DB2A

DB2D

z/OS

z/OS

Global

transaction

Connection 1

Connection 2

Re-routed connection SCA*

* XIDs: owning DB2member + indoubts

24

DB2 12 and XA: Example - End of Global Transaction

• XA_End sent to DB2A• DBAT becomes pooled XA DBAT• DISCONNX and DX in DIS THD (token 12)• Connection, from client, becomes inactive (token 10)

• XA_End sent to DB2D, re-routed to DB2A• DBAT becomes pooled XA DBAT• DISCONNX and DX in DIS THD (token 9) on DB2A• Connection, from DB2D (re-route) becomes inactive (token 11)

• Connection to DB2D remains active• Token 4 on DB2D

DB2A

DB2D

z/OS

z/OS

Global

transaction

XA_End

XA_End

Re-routed connection SCA*

* XIDs: owning DB2member + indoubts

25

Global Transaction Support - .NET Client Changes

• IBM data server clients:• enableDirectXA true|false

• enableDirectXA parameter can be set to true or false to specify whether XA support is on or off

• New global transaction support is not available in the gateway configuration

• IBM data server drivers• XA support is enabled by default for Microsoft Distributed Transaction

Coordinator (DTC) or Microsoft Component Services (COM+)

• Setting clientApplcompat=V12R1M500 in the db2dsdriver.config file is required

26

Global Transaction Support – Disable Centrally

• DSNZPARM: DDF_COMPATIBILITY• IDNTFY_V12_PRIOR_VER

• Indicates that DB2 11 behavior is enforced for remote applications

• Even if ACTIVATE FUNCTION LEVEL V12R1M500 or higher already issued

• Local applications may still exploit new functions

27

DRDA Fast Load

28

DRDA and Load: Current Situation

DB2 for z/OS provides DSNUTILU stored procedure First, transfer file to be used as LOAD utility input to z/OS LPAR

After file transfer is complete, call DSNUTILU, directing the LOAD utility to the transferred file via a TEMPLATE

Invoke LOAD utility from client

Load into table on DB2 for z/OS

Procedure complex, time consuming

29

DRDA Fast Load: DB2 12 for z/OS

• DB2 Call Level Interface (CLI) APIs and Command Line Processor (CLP) enhanced to support remote load to DB2 for z/OS server• Supported in all DB2 client packages

• Stream data to load process in continuous blocks

• Java applications also supported (no CLPPLUS support)

30

DRDA Fast Load: DB2 12 for z/OS

• Three file formats• DELIMITED (DEL) – sequential file with row and column delimiters

• EBCDIC, ASCII or UNICODE

• INTERNAL (INT) – created by running DB2 for z/OS UNLOAD with INTERNAL FORMAT option

• LOAD does not validate data to ensure DB2 internal format

• Can be created by client application (SAP) but internal format must be observed

• SPANNED (SPN) – sequential file for allowing loading of tables with LOB/XML columns

• Normal load process for SPANNED YES would expect RECFM=VBS file input

• RECFM=VBS files cannot be exchanged with a client

• Clients must build their own spanned format data stream or file where each record is preceded with an 8-byte integer containing length of stream for a row

31

DRDA Fast Load

• DB2 for z/OS: single DSNUTILU external:• New INDDN keyword: SYSCLIEN, for LOAD utility control statement

• CLP Command Syntax• ZLOAD FROM <filename> OF {INT | SPANNED | DEL [ALF | ENL | CRLF]}

[WITH <utilityid>] [MESSAGES <msg.filename>] USING <loadstmt>

• CLI application steps to invoke Fast Load:• Provide LOAD attributes:

• SQL statement

• Utility-ID

• Record delimiter

• Start the load and provide continuous blocks for LOAD

• End the LOAD and verify completion

32

DRDA Fast Load: CLI Statement Level Attributes

• SQLSetStmtAttr() API

• SQL_ATTR_DB2ZLOAD_UTILITYID

• 16-characters: A-Z upper and lower, 0-9, #, $, @, ₵, !, period (.)

• Default: ZLOADdddhhmmsstt

• SQL_ATTR_DB2ZLOAD_LOADSTMT (maximum 32704 characters)

• The LOAD statement supports full LOAD semantics

• SQL_ATTR_DB2ZLOAD_FILETYPE

• SQL_DB2ZLOAD_FILETYPE_DEL – delimited

• SQL_DB2ZLOAD_FILETYPE_INT – IBM internal

• SQL_DB2ZLOAD_FILETYPE_SPANNED – spanned (LOB/XML)

33

DRDA Fast Load: CLI Statement Level Attributes

• SQLSetStmtAttr() API

• SQL_ATTR_DB2ZLOAD_RECDELIM

• SQL_ATTR_DB2ZLOAD_RECDELIM_ALF – ASCII line feed

• SQL_ATTR_DB2ZLOAD_RECDELIM_ENL – EBCDIC newline

• SQL_ATTR_DB2ZLOAD_RECDELIM_CRLF – ASCII carriage return and line feed

• SQL_ATTR_DB2ZLOAD_RECDELIM_NONE – for IBM Internal or Spanned

• SQL_ATTR_DB2ZLOAD_BEGIN

• SQL_ATTR_DB2ZLOAD_END

• Control returned back to application when LOAD is complete and/or all LOB/file input data has been read from client if LOAD had an error (RETCODE>=8)

• DiagIdentifiers : SQLGetDiagField()

• ‘SQL_DIAG_DB2ZLOAD_RETCODE’

• ‘SQL_DIAG_DB2ZLOAD_LOAD_MSGS’

34

DRDA Fast Load: Benefits

• DSNUTILU load invocation overlapped with transfer of data blocks to LOAD.

• Performance test showed DRDA Fast Load as fast as LOAD alone• Significant elapsed time reduction

• Task which extracts data blocks and passes to LOAD utility can run at 100% zIIP offload using the goal of the distributed address space

35

Data Server Driver: Support for DB2 12 for z/OS

36

IBM Data Server Driver: Support for DB2 12 for z/OS

• New driver property: clientApplcompat• Character string

• DataSource

• Data.source.setClientApplcompat(“V12R1M500”);

• Connection URL or properties object:

• clientApplcompat=V12R1M500

• Applcompat V12R1M500 required for

• Workload balancing with Session Token support

• enableSysplexWLB=true OR

• clientAffinities

• When application connection closed, JCC client to perform clean disconnect

• Workload balancing for Global Transaction support

• enableXACleanTransaction

• false – enable dirty transaction mode (as today)

• true – enable clean transaction mode

37

IBM Data Server Driver: Support for DB2 12 for z/OS

• DB2 Data Server Driver 11.1 FP1 required

38

39

Thank You!

APPENDIX

40

DB2 12 and XA Support: Owning Member DB2A

41

DB2 12 and XA Support: Re-routing member DB2D

42

DB2 12 and XA Support: DB2A Display Active Threads

43

DB2 12 and XA Support: DB2A Display Inactive Threads

44

DB2 12 and XA Support: DB2D Display Active, Inactive Threads

45

PKGREL(BNDPOOL)• DB2 10 for z/OS introduced High Performance DBATs

• MODIFY DDF PKGREL introduced with two options:• COMMIT – connection inactivated, DBAT pooled

• BNDOPT – honor release setting of package bind• If RELEASE(DEALLOCATE), DBAT remained active = High Performance DBAT

• DB2 12 for z/OS and MODIFY DDF PKGREL• PKGREL(COMMIT)

• At commit, connection inactive and DBAT pooled, regardless of REL setting on package

• PKGREL(BNDOPT) – honor release setting on package bind. For high performance DBAT

• At commit, DBAT remains active on thread

• At connection termination, DBAT deallocated

• PKGREL(BNDPOOL) – honor release setting on package bind. For high performance DBAT

• At commit, DBAT remains active on thread

• At connection termination, DBAT pooled, can be used by other connections

• Retro fit to DB2 10 and DB2 11

46

Acknowledgements and Disclaimers Availability. References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates.

The workshops, sessions and materials have been prepared by IBM or the session speakers and reflect their own views. They are provided for informational purposes only, and are neither intended to, nor shall have the effect of being, legal or other guidance or advice to any participant. While efforts were made to verify the completeness and accuracy of the information contained in this presentation, it is provided AS-IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this presentation or any other materials. Nothing contained in this presentation is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software.

All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue growth or other results.

© Copyright IBM Corporation 2016. All rights reserved.

— U.S. Government Users Restricted Rights – Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.

IBM, the IBM logo, ibm.com, DB2, CICS and Parallel Sysplex are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol (® or TM), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at

•“Copyright and trademark information” at www.ibm.com/legal/copytrade.shtml

•Other company, product, or service names may be trademarks or service marks of others.

4747

Mark RaderIBMmrader@us.ibm.com

Session: A7Title: DB2 12 for z/OS: DRDA Enhancements

Please fill out your session

evaluation before leaving!