platform: db2 udb for z/os faster fetch & insert v8

1

Faster FETCH & INSERTV8 Multiple Row Processing

Christopher J. CroneSenior Technical Staff Member/IBM

Session: C10Wednesday May 25th, 12:30 – 1:40 PM

Platform: DB2 UDB for z/OS

2

IBM Software Group | DB2 Data Management Software

Multiple Row Insert and Multiple Row Fetch © 2003 IBM Corporation2

Presentation Topics

Host Variable ArraysMultiple-Row InsertMultiple-Row Fetch

DELETE WHERE CURRENT OFUPDATE WHERE CURRENT OF

GET DIAGNOSTICS

3

IBM Software Group

Confidential | Date | Other information, if necessary © 2002 IBM Corporation

Host Variable Arrays



Host Variable Arrays

Host variable array is an array in which each element of the array contains a value for the same column

Changes have been made to allow host variable arrays in:ƒ COBOLƒ PL/1ƒ C++ƒ NOTE: Assembler support is limited to cases where

USING DESCRIPTOR is allowed. Assembler pre-compiler does not recognize declaration of host variable arrays. The programmer is responsible for allocating storage correctly, etc.

Can only be referenced in multi-row fetch or insertIn general, arrays may not be arrays of structures

Some arrays of structures are supported, for instance, DB2 V8 does support arrays of structures for C varying length strings (C varying length strings are structures). DB2 V8 would not support an array of structures for say a structure that was created to fetch an entire row of data.

5



COBOL

This is an Example of the changed Syntax for a COBOL Numeric host variable –other declarations for things like strings, etc. have similar changes. For COBOL, the specification of the array size is determined by the “OCCURS dimension [TIMES]”clause.

6



PL/I

This is an example of how to declare a character (CHAR) or character varying (VARCHAR) string in PL/I.Again, other variable definitions have similar changes.

7



C/C++

This slide show the changes to C/C++ to declare a numeric array.

8



COBOLExample 1: Declare a CURSOR C1 and fetch 10 rows using a multi-row FETCH statement

01 OUTPUT-VARS.05 NAME OCCURS 10 TIMES.

49 NAME-LEN PIC S9(4)COMP-4 SY C.49 NAME-DATA PIC X(40).

05 SERIAL-NUMBER PIC S9(9)COMP-4 OCCURS 10 TIMES.

PROCEDURE DIVISION.

EXEC SQLDECLARE C1 CURSOR WITH ROWSET POSITIONING FORSELECT NAME, SERIAL# FROM CORPORATE.EMPLOYEE END-EXEC.

EXEC SQLOPEN C1 END-EXEC.

EXEC SQLFETCH FIRST ROWSET FROM C1 FOR 10 ROWS INTO :NAME,

:SERIAL-NUMBER END-EXEC.

This example shows a FETCH of 10 rows into two host-variable-arrays, NAME, and SERIAL-NUMBER.NAME is a VARCHAR, and SERIAL NUMBER is an INTEGER.

9



ODBC#define TC 10;SQLWCHAR SAVE_H1WCHR[TC][1025];SQLINTEGER LNSAVE_H1WCHR[TC];

/* Main Program */int main(){/* initialize data */wcscpy(SAVE_H1WCHR[0], (SQLWCHAR *)"abc 1");...

wcscpy(SAVE_H1WCHR[9], (SQLWCHAR *)“abc 10");

hstmt=0;

rc=SQLAllocStmt(hdbc, &hstmt);

rc=SQLBindParameter(hstmt, 1, SQL_PARAM_INPUT, SQL_C_WCHAR, SQL_DBCLOB1024, 0, SAVE_H1WCHR, (SQLINTEGER) 2050, (SQLINTEGER *) LNSAVE_H1WCHR);

/* Set number of rows to insert */rc=SQLParamOptions(hstmt, TC, NULL);

/* Insert rows into DBCLOB column via SQLBindParameter */rc=SQLPrepareW(hstmt,(wchar_t *)

"INSERT INTO TABLECU (C1) VALUES (?)",SQL_NTS);

rc=SQLExecute(hstmt);

rc=SQLTransact(henv, hdbc, SQL_COMMIT);

} /* End Main */

In this ODBC example, we insert 10 rows into table “TABLECU” using and ODBC Array Insert API.In particular, note that SQLParamOptions, the number of rows inserted (TC in this case) is specified.Please note that in this example the error checking for each statement has been removed (rc is set as each statement is processed, but the error checking of RC has been omitted for brevity).

10

IBM Software Group


Multiple-Row Insert

11



Different Forms of INSERTƒ INSERT via VALUES is used to insert a single row into the table or

view using values provided or referencedƒ INSERT via SELECT is used to insert one or more rows into table or

view using values from other tables or viewsƒ INSERT via VALUES… FOR "n" ROWS form is used to insert multiple

rows into table or view using values provided in host variable array

FOR "n" ROWSƒ For static, specify FOR "n" ROWS on INSERT statement

(for dynamic INSERT, you may also specify FOR "n" ROWS on EXECUTE statement)

ƒ Maximum value of n is 32767 specified as host-variable, parameter marker, or literal value

ƒ Input provided with literal, host variable, or host variable array --each array represents cells for multiple rows of a single column

VALUES… FOR “n” ROWS clause allows specification of multiple rows of

data ƒ Host variable arrays used to provide values for a column on INSERT ƒ Example: VALUES (:hva1, :hva2) FOR 10 ROWS

Multiple Row INSERT

New syntax is added to allow a “FOR ‘n’ ROWS” clause to be added to INSERT. This clause allows specification of the number of rows to be inserted. The clause may be specified on the INSERT statement (either static or dynamic SQL), or as part of the EXECUTE statement (dynamic SQL only).

12



Multiple-Row Insert

Changed Statementsƒ INSERTƒ PREPAREƒ EXECUTE

Changes were made to INSERT, PREPARE, and EXECUTE to support MRI.

13



INSERT

Here is part of the new INSERT syntax diagram. Note the multiple-row-insertclause

14



INSERT (cont)

In this segment of the INSERT syntax diagram, we show changes to the VALUES clause to allow specification of host-variable-arrays, and optionally, the number or rows to be inserted.We’ll mostly focus on the “FOR ‘n’ ROWS” clause next, We’ll talk atomicity later.

15



INSERT Example 1Insert a variable number of rows using host variable arrays for columnvalues. Assume that the table T1 has one column and that a variable(:hv) number of rows of data are to be inserted into T1 table.

EXEC SQL INSERT INTO T1VALUES (:hva :hvind) FOR :hv ROWSATOMIC;

In this example, :hva represents the host variable array and :hvind represents the array of indicator variables

In this example, :hva represents the host variable array and :hvindrepresents the array of indicator variables

16



INSERT Example 2

Insert 10 rows into a table T2, and return the rows that have been inserted using a multiple row fetch statement.

DECLARE CS1 ASENSITIVE SCROLL CURSOR WITH RETURN WITH ROWSET POSITIONING FOR SELECT T2.C1, T2.C2 FROM FINAL TABLE

(INSERT INTO T2 VALUES (:hvai1 :hvindi1, :hva2 :hvinid2) FOR 10 ROWS);

EXEC SQL OPEN CS1; /* INSERT OCCURS HERE */EXEC SQL FETCH FIRST ROWSET FROM CS1

FOR 10 ROWS INTO :hvao1 :hvindo1, :hvao2 :hvindo2

In this example, 10 rows of data are inserted into T2 (columns C1 and C2). The rows are inserted as part of a SELECT … FROM FINAL TABLE statement. This statement was declared and associated with CURSOR CS1. When CURSOR CS1 is opened, the actual INSERT will occur, and the rows that are inserted will be materialized to a temporary table.The input values are specified in host-variable-arrays hvai1 and hvai2. Indicator arrays are specified by hvindi1 and hvindi2.After the OPEN, the rows that have been inserted are then fetched back by the application using a rowset cursor. These output values are placed in host variable arrays :hvao1 and hvao2. Indicator arrays are specified by hvindo1 and hvindo2.

17



Assume that table T3 has two columns: C1 (SMALLINT) and C2(INTEGER)The application allocates two arrays, :hva1 with 5 elements, and :hva2 with 10 elements

:hva1 :hva2

INSERT INTO T3 (C1, C2) VALUES (:hva1, :hva2) FOR 5 ROWS;

INSERT Example 3

In this example, hva1 has 5 elements and hva2 has 10 elements.5 rows are specified to be inserted using the “FOR ‘n’ ROWS” clauseIf more than 5 rows were specified on the “FOR ‘n’ ROWS” clause, an error would have occurred. For INSERT, the host variable arrays must be at least as large as specified by the “FOR ‘n’ ROWS” clause.

18



Assume that table T3 has three columns: S1 SMALLINT, I1 INTEGER, and M1 CHAR(8)The application allocates one array, :hva2 with 10 elements. :hv1 is a scalar host variable and the value for M1 is a special register In this example, hv1, and CURRENT MEMBER are used for each row of data, the values for C2 are obtained from host-variable-array :hva2

:hv1 :hva2INSERT INTO T3 VALUES (:hv1, :hva2,Current Time) FOR 10 ROWS -- Assume CURRENT MEMBER = ‘DB2A’

INSERT Example 4

M1I1S1

DB2A105

DB2A95

DB2A85

DB2A75

DB2A65

DB2A55

DB2A45

DB2A35

DB2A25

DB2A155 12345678910

Result of Insert

In this example, we have an insert into table T3 a scalar host variable hv1, which is used for C1 for every row of the insert, and a host-variable-array with 10 elements for column C2, and a literal value ‘ABCD’ which is also used for every row of the insert.

19



Local INSERT Flow

1900

1700

1300

1204

0

0

0

0

1900

1700

1300

1204 … …

SQLDAID = SQLDASQLDABC = 104SQLN = 2SQLD = 2SQLTYPE[1] = 496 SQLTYPE[2] = 496SQLLEN[1] = 4 SQLLEN[2] = 4SQLDATA[1] = x00001000 SQLDATA[2] = x00003000SQLIND[1] = x00002000 SQLIND[2] = x00000000SQLNAME[1] = x‘0008000000010004’ SQLNAME[2] =x‘0008000000020000’

x1000 x2000 Internal Buffers

4

Table

4

x3000

The host variable array starts at x1000The indicator array starts at x2000The value N is locates at x3000Runtime will first locate the value for N and move it into the DB2 (ADMF) address spaceRuntime will then move the values for the indicator and column (assuming the indicator is not null)Runtime will then drive DM INSERTRuntime will then move the next value for the indicator and column into the DB2 address spaceRuntime will then drive DM again….Note that the SQLNAME values for 1 and 2 are set to x’0008’ (the length) of information provided in this field.The values x‘0000’ (in black) represent the CCSID – not specified in this case, so the application encoding bind option will be used.Next, for SQLNAME[1], we have x’0001’ (in Red) which indicates that the SQLVAR contains an array entry, followed by x’0004’ (in Blue) which represents the dimension of the array.Finally for SQLNAME[2], we have x’0002’ (in Red) which indicates the SQLVAR contains an entry for “N”, followed by x’0000’ (in Blue) which is default information for this type of entry.

20



Distributed Multiple Row INSERT FlowApplicationC1 C2

Client Com-Buffer

IntermediateBuffer

Table Page

ServerCom-Buffer

Distributed Chained INSERT FlowApplicationStmt Data

Client Com-Buffer

IntermediateBuffer

Table Page

ServerCom-Buffer

For a distributed flow, the data is blocked up at the requestor and moved to the server.The value for N will flow as part of the statement metadata.Runtime will then move the values for the first row from the com buffer to the intermediate bufferRuntime will then drive DM INSERTRuntime will then move the values for the next row from the com buffer to the intermediate bufferRuntime will then drive DM again….

21



ATOMIC -vs- NOT ATOMIC

ATOMICƒTraditional behaviorƒAll rows being inserted must successfully be inserted

NOT ATOMIC CONTINUE ON SQLEXCEPTIONƒInsert rows that are successfulƒReject rows that are not successful

GET DIAGNOSTICS can be used to determine which rows were not successful

ƒSQLCODE will indicate if all failed, all were successful or at least one failed

A major consideration of ATOMIC vs NOT ATOMIC is the amount of data you are inserting -- inserting 32K rows into a table whose rows are 32K bytes long (the row is in a 32K page and 1 row/page) consumes 1G of space in the application and would log > 1G of data, so rollback could be painful.NOT ATOMIC CONTINUE ON SQLEXCEPTION allows some rows to be successfully inserted into a table, and failing rows to be indicated for possible further processing (For example, SQLCODE -803).GET DIAGNOSTICS should be used to determine the failing rows.

22



ATOMIC -vs- NOT ATOMIC with TriggersTrigger Behavior on Multiple Row Insert

ATOMICƒThe inserts are processed as a single statement. ƒAny statement level triggers fire once for the statement, and the transition tables will include all of the rows that were inserted.

NOT ATOMIC CONTINUE ON SQLEXCEPTIONƒInserts are processed separately. ƒAny statement level triggers are processed for each row that is insertedƒTransition tables include the individual row that is inserted. ƒWhen errors are encountered with this option in effect, processing continues, and some of the specified rows will not beinserted.

ƒIn this case, if an insert trigger is defined on the underlying base table, the trigger transition table will only include rows that were successfully inserted.

A major consideration of ATOMIC vs NOT ATOMIC is the amount of data you are inserting -- inserting 32K rows into a table whose rows are 32K bytes long (the row is in a 32K page and 1 row/page) consumes 1G of space in the application and would log > 1G of data, so rollback could be painful.NOT ATOMIC CONTINUE ON SQLEXCEPTION allows some rows to be successfully inserted into a table, and failing rows to be indicated for possible further processing (For example, SQLCODE -803).GET DIAGNOSTICS should be used to determine the failing rows.

23



PREPARE

For Prepare, we have new attributes. FOR MULTIPLE ROWS or FOR SINGLE ROWS and atomicity may be specified on Prepare as attributes.

24



EXECUTE

Changes to EXECUTE allow specification of host-variable-arrays and the “FOR ‘n’ROWS” clause.The “FOR ‘n’ ROWS” clause may only be specified once, either on the Dynamic INSERT statement, or the EXECUTE statement. If the “FOR ‘n’ ROWS” clause is specified in both places, then an error (negative SQLCODE) will occur.

25



PREPARE and EXECUTE - Example

Assume that the prog table has 9 columns. Prepare and execute a dynamicINSERT statement which inserts 5 rows of data into the prog table.

stmt = 'INSERT INTO prog (iwhid, updated_by, update_ts, name, short_description,orderNo, parmData, parmDataLong, VWProgKey)

VALUES (?,?,?,?,?,?,?,?,?)';

attrvar = ’FOR MULTIPLE ROWS ’;NROWS = 5;

EXEC SQL PREPARE ins_stmt ATTRIBUTES :attrvar FROM :stmt;

EXEC SQL EXECUTE ins_stmt FOR :NROWS ROWSUSING :V1,:V2,:V3,:V4,:V5,:V6,:V7,:V8,:V9;

In this example, each host variable in the USING clause represents an array of values for the corresponding column of the target of the INSERT statement.

In this example, we have a statement string “stmt” which contains an INSERT statement.INSERT attributes are specifications are specified on PREPARE and the EXECUTED statement includes the “FOR ‘n’ ROWS” clause.

26



SQLDA must contain a valid description of the host variable arrays or buffers which contain the values to be inserted

ƒ Each SQLVAR describes a host variable or host variable array which represents a buffer which contains value(s) for a column of target table

ƒ SQLDA must have enough storage to contain SQLVAR for each target column for which values are provided, plus an additional SQLVAR entry for use by DB2 for z/OS

ƒ Prior to the multi-row insert, the SQLDA fields must be set correctly to include number of SQLVAR occurrences, number of variables used, pointer to arrays, indicator variables etc.

INSERT SQLDA Considerations

If you code statements with “USING DESCRIPTOR” instead of “USING” host variables, there are some changes that you might need to make.Please review the SQL Reference, Appendix C, for more information regarding SQLDA fields that must be filled in for MRI. In particular, pay attention to the values that must be set in the SQLNAME field for each SQLVAR.

27



MRI Performance ConsiderationsUp to 30% faster INSERT performance

Performance improvement largely due to savings of API costsSavings flattens out quickly, for example, savings (as a percentage) was equal

• For 100 Rows, 500 Rows, 1000 Rows…Reasonable “n” for MRI is about 200 – no additional savings above that, and

downside (rollback) increases.Distributed MRI performance up to 70% Elapsed Time and 50% Server CPU time reductions seenPerformance variable based on

• Number of rows INSERTed• Number of columns INSERTed• Number of INDEXes on table• Class 2 accounting (on or off) – savings larger is Class 2 is on

Note: Don’t use MRI to INSERT 1 row due to overhead to set up for MRI

Similar improvement with UPDATE and DELETE WHERE CURRENT OF when updating/deleting the entire rowset. This is in addition to the savings provided by MRF

As always, when talking about performance, Your Mileage May Vary (YMMV). These are example numbers that we have seen in our testing.

28

IBM Software Group


Multiple-Row Fetch

29



Multiple-Row Fetch

Changed Statementsƒ DECLARE CURSORƒ OPEN CURSORƒ ALLOCATE CURSORƒ FETCHƒ Positioned UPDATE (UWCO)ƒ Positioned DELETE (DWCO)

Changes were made to DECLARE CURSOR, OPEN CURSOR, ALLOCATE CURSOR, DESCRIBE CURSOR, FETCH, UPDATE WHERE CURRENT OF (UWCO), and DELETE WHERE CURRENT OF (DWCO) for MRF.The changes for OPEN and ALLOCATE Cursor are shown in the back of this presentation.

30



DECLARE CURSOR

DECLARE CURSOR adds a new “WITH ROWSET POSITIONING” clause. This clause must be used for MRF cursors, and may be used for non-MRF cursors.Row positioning FETCH statements may be used with cursor declared with the “WITH ROWSET POSITIONING” clause, or the “WITHOUT ROWSET POSITIONING” clause. Rowset positioning FETCH statements may only be used with cursors declared with the “WITH ROWSET POSITIONING” clause.

31



DECLARE CURSOR - Example

Declare C1 as the cursor of a query to retrieve a rowset from the table DEPT.

The prepared statement is MYCURSOR

EXEC SQL DECLARE CURSOR C1 CURSORWITH ROWSET POSITIONINGFOR MYCURSOR;

Rowset positioning specifies whether multiple rows of data can be accessed as a rowset on a single FETCH statement –default is WITHOUT ROWSET POSITIONING

This example demonstrates the new “WITH ROWSET POSITIONING” clause on DECLARE CURSOR.

32



FETCH

In this syntax diagram, you will see that rowset positioned fetches have been added that correspond to the row-positioned fetches.

33



FETCH (cont)

Also added to FETCH is the specification of a “FOR ‘n’ ROWS” clause, and the ability to fetch data into host-variable-arrays.

34



Rowsets

A group of rows for the result table of a query which are returned by a single FETCH statement

Program controls how many rows are returned (i.e., size of the rowset)ƒ Can be specified on the FETCH statement (maximum rowset size is

32767)

Each group of rows are operated on as a rowset

Ability to intermix row positioned and rowset positioned fetches when a cursor is declared WITH ROWSET POSITIONING

FETCH FIRST ROWSET STARTING AT ABSOLUTE 10 FROM CURS1

FOR 6 ROWS INTO :hva1, :hva2;

A ROWSET is a grouping of rows. When a cursor is positioned on a rowset, all locks (if any) are held on all rows of the rowset.

FOR n ROWS with FETCH FIRST n ROWS ONLY these two clauses may be used together. FETCH FIRST n ROWS ONLY dominates.

Remote applications with updateable cursors should see an improvement in performance of fetching on these cursors. The functional changes introduced with this line item will allow blocking on cursors where no blocking was allowed before.

35



Determining rowset size

If FOR n ROWS is NOT specified and cursor is declared for rowset positioning..

Size of rowset will be the same as the previous rowset fetch as long as ƒ It was the previous fetch for this cursorƒ Or the previous fetch was a FETCH BEFORE or FETCH AFTER and the fetch

before that was a rowset fetch

Else rowset is 1

FETCH FIRST ROWSET FOR 5 ROWS FETCH NEXT ROWSETFETCH NEXTFETCH NEXT ROWSETFETCH NEXT ROWSET FOR 3 ROWSFETCH BEFORE FETCH NEXT ROWSET

Returns 5 rowsReturns the next 5 rowsReturns a single rowReturns a single rowReturns 3 rowsReturns 0 rowsReturns 3 rows

The size of the rowset is determined explicitly (when specified on the FETCH statement), or implicitly.

36



FETCH examples

EXAMPLE 1:Fetch the previous rowset and have the cursor positioned on that rowset

EXEC SQL FETCH PRIOR ROWSET FROM C1 FOR 3 ROWS INTO...

-- OR --EXEC SQL

FETCH ROWSET STARTING AT RELATIVE -3 FROM C1 FOR 3 ROWS INTO...

EXAMPLE 2:Fetch 3 rows starting with row 20 regardless of the current position of the cursor

EXEC SQL FETCH ROWSET STARTING AT ABSOLUTE 20

FROM C1 FOR 3 ROWS INTO...

Here we have a couple of examples using the new FETCH syntax.In example 1, we have two statements that fetch the three rows prior to the current rowset.In the second example, we have a statement that will fetch three rows starting at row 20.



Partial Result Sets

If you fetch beyond the end of the result set, you will receive an end of data condition

ƒ i.e., When there are only 5 rows left in result table and you request FETCH NEXT ROWSET FOR 10 ROWS, 5 rows will be returned - SQLCODE +100

ƒ SQLERRD(3) will contain the number or rows returnedƒ This includes where FETCH FIRST n ROWS ONLY has been

specified

If you fetch beyond the beginning of the result set, you will receive an end of data condition

ƒ i.e., if you are positioned on rows 3,4,5,6, and 7, and you request FETCH PRIOR ROWSET FOR 10 ROWS, 2 rows will be returned (Rows 1 and 2) - SQLCODE +20237

ƒ SQLERRD(3) will contain the number or rows returned

Partial results sets are possible if an application fetches rows beyond the boundaries of the result set.SQLERRD 3 will contain the number of rows returned.SQLCODE +100 will be returned when an application fetches beyond the end of the data.SQLCODE +20237 will be returned when an application fetches beyond the beginning of the data.



Fetching Beyond the Result Set ABSOLUTE or RELATIVEIf you fetch beyond the end of the result set, or beyond the beginning of the result set, you will receive an end of data condition

ƒ Assume you are positioned on row 5 in a result set with 10 rows.

ƒFETCH ROWSET STARTING AT ABSOLUTE 15ƒFETCH ROWSET STARTING AT RELATIVE -7

ƒ No rows will be returned - SQLCODE +100ƒ SQLERRD(3) will contain 0ƒ Cursor position will be either “BEFORE” or “AFTER” depending

on the direction of the FETCH.

No results are returned if an application fetches rows beyond the boundaries of the result set using FETCH ABSOLUTE, or FETCH RELATIVE, and the starting position requested would be before the beginning or after the end of the result set.SQLERRD 3 will contain the number of rows returned – 0 in all these cases.SQLCODE +100 will be returned.The cursor will be positioned either “BEFORE” or “AFTER” depending on the direction of the FETCH.

39



Rowset Positioned Fetches

CUST_NO CUST_TYP CUST_NAME

1 P Ian2 P Mark3 P John4 P Karen5 P Sarah6 M Florence7 M Dylan8 M Bert9 M Jo

10 R Karen11 R Gary12 R Bill13 R Geoff14 R Julia15 R Sally

FETCH FIRST ROWSET FOR 3 ROWS

FETCH NEXT ROWSET

FETCH ROWSET STARTING AT ABSOLUTE 8

FETCH NEXT ROWSET FOR 10 ROWSPartial Result Set

Result table

This example shows how the cursor is positioned after various ROWSET positioned FETCH statements. The last example “FETCH NEXT ROWSET FOR 10 ROWS” would result in 5 ROWS returned. SQLERRD(3) would be set to 5, and SQLCODE + 100 returned.Note : The cursor is positioned on ALL rows in current rowset



Row and Rowset Positioned Fetches

CUST_NO CUST_TYP CUST_NAME

1 P Ian2 P Mark3 P John4 P Karen5 P Sarah6 M Florence7 M Dylan8 M Bert9 M Jo

10 R Karen11 R Gary12 R Bill13 R Geoff14 R Julia15 R Sally

FETCH BEFORE

FETCH FIRST

FETCH ABSOLUTE 4

FETCH NEXT ROWSETFOR 3 ROWS

FETCH NEXT Relative to first row in current rowset

FETCH ABSOLUTE 20FETCH LAST

Result table

With row-positioned, cursor is positioned on single row of data - host variables assigned values unless fetch before or after

When cursor is positioned on a rowset, a single row fetch will act relative to the first row in the rowset, i.e., fetch current would return the first row in the current rowset

Note that the FETCH ABSOLUTE 20 would result in SQLCODE +100, the cursor position would be placed in the “AFTER” position.

41



Distributed Multiple Row Fetch FlowApplicationC1 C2

Client Com-Buffer

IntermediateBuffer

Table Page

ServerCom-Buffer

DiagnosticInfo SQLCA Diagnostic

InfoDiagnostic

Info

Limited Block Fetch FlowApplicationC1 C2

Client Com-Buffer

IntermediateBuffer

Table Page

ServerCom-Buffer

DiagnosticInfo Diag Info Diag Info

SQLCA

SQLCA

SQLCA

In this flow, data from a table is sent to a client program a row at a time.In this flow, Columns C1, and C2 are placed into the Communications buffer at the server.The buffer then flows as one piece to the client (the blocksize is determined by the client using the FOR n ROWS clause on FETCH).At the client, the data is then returned to the application in column-array format.In this picture, the Blue and Teal blocks represent column information, Red blocks represent diagnostic information,And Green blocks represent control information.Note that there is a 1 byte diagnostic area that flows with each row of data. This diagnostic area is always NULL. Diagnostic information for the statement flows after the data.The application will receive the SQLCA as normal. Extended diagnostics are available at via the GET DIAGNOSTICS command.

42



Local FETCH Flow

1900

1700

1300

1204

0

0

0

0

1900

1700

1300

1204 … …

SQLDAID = SQLDASQLDABC = 104SQLN = 2SQLD = 2

SQLTYPE[1] = 496SQLLEN[1] = 4 SQLDATA[1] = x00001000SQLIND[1] = x00002000SQLNAME[1] = x‘0008000000010004’

x1000 x2000 Internal Buffers ‘N’ Table

4

This example is to demonstrate how the processing of a multi-row FETCH proceeds.The host variable array starts at x1000The indicator array starts at x2000The value N is locates at x3000Runtime will first locate the value for N and move it into the DB2 address spaceRuntime will then move the values for the indicator and column (assuming the indicator is not null)Runtime will then drive DM INSERTRuntime will then move the next value for the indicator and column into the DB2 address spaceRuntime will then drive DM again….Note that SQLNAME[1] is set to specify a length of 8 (green), no CCSID override (black x’0000’), an indication that SQLVAR[1] is an array (red), and the size of the array (blue).

43



MRF Performance Considerations

Up to 50% faster FETCH performance Performance improvement largely due to savings of API costsPerformance variable based on

• Number of rows fetched• Number of columns fetched• Class 2 accounting (on or off) – savings larger is Class 2 is on

ExamplesDSNTEP4

35% improvement on FETCH of 1000 rows with 5 cols and 20 cols

DSNTIAUL50% improvement on FETCH of 10000 rows with 5 cols and 20 cols

Up to 50% reduction in CPU cost for LBF –vs- V7

As always, when talking about performance, Your Mileage May Vary (YMMV). These are example numbers that we have seen in our testing.



Locking and Isolation Levels

Cursor will be positioned on all rows in current rowset

Locks will be held on all rows in rowset depending on isolation level and whether a result table has been materialized

As normal, these factors affect whether you will see changes made to the table following open cursor

Also affects whether you refetch the same rows when issuing a FETCH CURRENT to refetch current rowset

Isolation levels can greatly affect the results of your processing of rowset cursors, especially if you are using dynamic scrollable cursors.In general, locks will be held on all rows in the rowset – if a lock is held on one row, it will be held on all rows. CS current data no, and UR may not hold locks.Isolation levels can affect the results of subsequent fetches. For example, with ISO RR, a FETCH CURRENT ROWSET will always return the same rowset. However with other isolation levels, you may not see the same rows.

45



Using Static Scrollable Cursors

When scrolling between rowsets:

With insensitive fetches, updates by the application itself could cause changes and holes

With sensitive fetches, updates by other applications could cause changes and holes

For example, FETCH PRIOR ROWSET may return update or delete holes in place of rows that were fetched before

Row contents can change between fetches

With ROWSET fetches, you are more likely to encounter some of the issues that may occur if you fetch using a static scrollable cursor. The way an application handles these issues, such as hole rows, is different for a rowset FETCH due to the nature of the way data is returned.



If using static scrollable cursors...

CUSTNO NULLABLE_COL IND_VAR CUST_TYPE

1000 M P 2000 -1 B

-34000 F P

As holes may occur, ensure at least one indicator variable is defined for a row

ƒ If no nullable columns exist add an indicator variable for at least one columnƒ If multiple nullable columns exist each indicator variable will be updated if a

hole is found

New value of -3 indicates hole

SQLCODE +222 will also be returned (via GET DIAGNOSTICS)

If you fetch from a static scrollable cursor, a new indicator value, -3, is used to reflect holes. With row-positioned FETCH statements, this was not needed because only a single rows was returned, and the SQLCODE +222 was enough information to indicate the hole. With a MRF, -3 is used to indicate which rows have holes. The -3 will be returned for all indicator-variable-arrays that are specified. If at least one indicator is not provided, a negative SQLCODE will result.



Locks will be held on base table for entire rowset with ISO CS, RR and RS

Starting point and contents of rowsets may change when scrollingback and forth

Refetching current rowset may return different rowsƒ If ISO(RR), other applications cannot affect your rowsetƒ Your application can still affect your rowset

FETCH PRIOR ROWSET will return the previous n rows that qualify from the start of the current cursor position

ƒ Therefore n rows will be returned as long as start of rowset is not reached

Considerations with Dynamic Scrollable Cursors

Dynamic scrollable cursors are not supported for queries which result in materialized result tables (i.e., a query containing an ORDER BY, which is not supported by an index). Consequently locks will always be held on the base table according to normal locking rules.

The contents of a rowset are a point in time statement. A subsequent FETCH may reflect different rows (Except ISO RR). For example, FETCH NEXT ROWSET, followed by FETCH PRIOR ROWSET may not result in the same data being returned. Even FETCH CURRENT ROWSET may result in different values being returned. If your application cannot handle these sorts of changes, do not use DYNAMIC SCROLLABLE CURSORS, use STATIC SCROLLABLE CURSORS.



Multiple Row FETCH/INSERT --DRDA Considerations

For remote client one rowset returned in a network request – this is in effect the blocksize

Updateable cursors will still be blocked.

Supported by any requester or server that supports DRDA Level 3

DB2 for z/OS V8DB2 LUW V8 FP 4 ODBC client

Limit of 10 megabyte buffer for distributed MRF/MRI

Distributed support is only in ODBC

Embedded support for distributed is not available yet

One advantage that rowset cursors will enjoy is that for updateable cursors in V7 and below the protocol is a one row at a time (no blocking) protocol.

With rowset cursors, the blocksize will be the rowset size. So blocking is now possible for updateable cursors.

49



Positioned DELETE

Positioned Update

UPDATE WHERE CURRENT OF (UWCO) and DELETE WHERE CURRENT OF (DWCO) have been modified.New syntax has been added to allow you to delete a specific row within a rowset.Use of existing syntax will cause entire rowset to be affected.

50



Positioned DELETE Example

Assuming cursor CS1 is positioned on a rowsetconsisting of 10 rows of table T1:

The following DELETE statement could be used to DELETE all 10 rows in the rowset

EXEC SQL DELETE FROM T1 WHERE CURRENT OF CS1;

The following DELETE statement could be used to DELETE the 4th row of the rowset.

EXEC SQL DELETE FROM T1 WHERE CURRENT OF CS1 FOR ROW 4 OF ROWSET;

The first example will delete all the rows of the current rowset.The second example will delete the 4th row of the current rowset.

51



Positioned UPDATE ExampleAllows positioned UPDATE or DELETE to be used on a rowsetcursor

UPDATE T1 SET COL1='ABC‘WHERE CURRENT OF CS1 FOR ROW :hv OF ROWSET;

Assuming cursor CS1 is positioned on a rowset consisting of 10 rows of table T1, the following UPDATE statement could be used to update all 10 rows in the rowset

EXEC SQL UPDATE T1SET C1 = 5WHERE CURRENT OF CS1;

The first example will update the row specified by :hv for the current rowset.The second example will update every row of the current rowset.



Positioned UPDATE/DELETE considerations

It is possible for another application process to update or delete a row in the base table of the SELECT statement so that the specified row of the cursor no longer has a corresponding row in the base table

ƒ If the cursor is non-scrollable, this could happen if the cursor is ISOLATION(CS) CURRENTDATA(NO) and lock avoidance succeeds

ƒ If the cursor is static scrollable, this could happen, since the result set is materialized into a temporary table and all underlying locks are released (unless ISOLATION(RR) or ISOLATION(RS))

ƒ If the cursor is dynamic scrollable, this could happen if the cursor is ISOLATION(CS) CURRENTDATA(NO) and lock avoidance succeeds.

These considerations apply to row-positioned cursors also. They are brought up here because with a rowset-positioned fetch, you are more likely to see these situations.

53

IBM Software Group


GET DIAGNOSTICS

The syntax changes for GET DIAGNOSTICS are shown in the Reference Material section at the back of this presentation.



GET DIAGNOSTICS statementEnables more diagnostic information to be returned than can be contained in SQLCAReturns SQL error information

ƒ for overall statementƒ for each condition (when multiple conditions occur)

Supports SQL error message tokens greater than 70 bytes (SQLDA Limitation)

INSERT INTO T1 VALUES (:array) FOR 5 ROWS ;GET DIAGNOSTICS :ERR_COUNT = NUMBER;

DO i = 1 TO ERR_COUNT; GET DIAGNOSTICS FOR CONDITION :i

:rc = RETURNED_SQLCODE;END;

New statement that enables more diagnostic information than can be contained in the SQLCA to be returned

Diagnostic information is a superset of the information available in the SQLCA

GET DIAGNOSTICS is the "way of the future"

55



GET DIAGNOSTICS - Example

In an application, use GET DIAGNOSTICS to handle multiple SQL Errors.

long numerrors,counter;char retsqlstate [5 ];

EXEC SQL GET DIAGNOSTICS :numerrors = NUMBER;for (i=1;i <numerrors;i++)

{EXEC SQL GET DIAGNOSTICS CONDITION :i :retsqlstate = RETURNED_SQLSTATE;

printf("SQLSTATE =%s",retsqlstate);}

Execution of this code segment, will set and print retsqlstate with the SQLSTATEfor each error that was encountered in the previous SQL statement.

In this example, we have a piece of C/C++ code to print the returned SQLSTATE for each condition.



Example – Get Diagnostics ALL

Insert (not atomic) 10 Rows into a table with a Unique Index, two rows Fail, 8 rows succeed, use GET DIAGNOSTICS to determine which rows failed.GET DIAGNOSTICS :hv ALL STATEMENT, CONDITION;

Statement information

NUMBER=3;ROW_COUNT=08;

This example returns the number of rows processed by the previous INSERT statement (ROW_COUNT), and the number of diagnostic conditions (NUMBER).



Example – Condition Number 1

CONDITION_NUMBER=1;DB2_RETURNED_SQLCODE=-253;RETURNED_SQLSTATE=22529; DB2_ERROR_CODE1=-500;DB2_ERROR_CODE2=13172739;DB2_ERROR_CODE3=8;DB2_ERROR_CODE4=13817814;DB2_SQLERRD1=-500;DB2_SQLERRD2=13172739;DB2_SQLERRD3=8;DB2_SQLERRD4=13817814;DB2_SQLERRD5=-490143744;DB2_INTERNAL_ERROR_POINTER=-500;DB2_MODULE_DETECTING_ERROR=DSNXRINS;MESSAGE_ID=DSN00253E ;SERVER_NAME=STLEC1;MESSAGE_TEXT=A NON-ATOMIC STATEMENT SUCCESSFULLY COMPLETED FOR SOME OF THE REQUESTED ROWS, POSSIBLY WITH WARNINGS, AND ONE OR MORE ERRORS;

This example shows how to return all diagnostic information using the ALL STATEMENT clause.This slide shows that the first diagnostic condition contains the information for SQLCODE -253.



Example – Condition Number 2CONDITION_NUMBER=2;DB2_RETURNED_SQLCODE=-803;RETURNED_SQLSTATE=23505;DB2_ROW_NUMBER=010;DB2_ERROR_CODE1=-110;DB2_ERROR_CODE2=13172739;DB2_ERROR_CODE4=13817814;DB2_SQLERRD1=-500;DB2_SQLERRD2=13172739;DB2_SQLERRD3=8;DB2_SQLERRD4=13817814;DB2_SQLERRD5=-490143744;DB2_INTERNAL_ERROR_POINTER=-110;DB2_MODULE_DETECTING_ERROR=DSNXRINS;MESSAGE_ID=DSN00803E ;SERVER_NAME=STLEC1;DB2_ORDINAL_TOKEN_1 =I1;DB2_ORDINAL_TOKEN_2 =000000020A;MESSAGE_TEXT=AN INSERTED OR UPDATED VALUE IS INVALID BECAUSE INDEX IN INDEX SPACE I1 CONSTRAINS COLUMNS OF THE TABLE SO NO TWO ROWS CAN CONTAIN DUPLICATE VALUES IN THOSE COLUMNS. RID OF EXISTING ROW IS X''000000020A'';

This example shows how to return all diagnostic information using the ALL STATEMENT clause.This slide shows that the second diagnostic condition contains the information for SQLCODE -803.



Example – Condition Number 3CONDITION_NUMBER=3;DB2_RETURNED_SQLCODE=-803;RETURNED_SQLSTATE=23505;DB2_ROW_NUMBER=009;DB2_ERROR_CODE1=-110;DB2_ERROR_CODE2=13172739;DB2_ERROR_CODE4=13817814;DB2_SQLERRD1=-500;DB2_SQLERRD2=13172739;DB2_SQLERRD3=8;DB2_SQLERRD4=13817814;DB2_SQLERRD5=-490143744;DB2_INTERNAL_ERROR_POINTER=-110;DB2_MODULE_DETECTING_ERROR=DSNXRINS;MESSAGE_ID=DSN00803E ;SERVER_NAME=STLEC1;DB2_ORDINAL_TOKEN_1 =I1;DB2_ORDINAL_TOKEN_2 =000000020B;MESSAGE_TEXT=AN INSERTED OR UPDATED VALUE IS INVALID BECAUSE INDEX IN INDEX SPACE I1 CONSTRAINS COLUMNS OF THE TABLE SO NO TWO ROWS CAN CONTAIN DUPLICATE VALUES IN THOSE COLUMNS. RID OF EXISTING ROW IS X''000000020B'';

This example shows how to return all diagnostic information using the ALL STATEMENT clause.This slide shows that the third diagnostic condition contains the information for SQLCODE -803.



Example – What you really see:hv = ‘NUMBER=3;ROW_COUNT=08;CONDITION_NUMBER=1;DB2_RETURNED_SQLCODE=-253;RETURNED_SQLSTATE=22529;DB2_ERROR_CODE1=-500;DB2_ERROR_CODE2=13172739; DB2_ERROR_CODE3=8;DB2_ERROR_CODE4=13817814;DB2_SQLERRD1=-500;DB2_SQLERRD2= 13172739;DB2_SQLERRD3=8;DB2_SQLERRD4=13817814;DB2_SQLERRD5=-490143744; DB2_INTERNAL_ERROR_POINTER=-500;DB2_MODULE_DETECTING_ERROR= DSNXRINS;MESSAGE_ID=DSN00253E ;SERVER_NAME=STLEC1;MESSAGE_TEXT=A NON-ATOMIC STATEMENT SUCCESSFULLY COMPLETED FOR SOME OF THE REQUESTED ROWS, POSSIBLY WITH WARNINGS, AND ONE OR MORE ERRORS;CONDITION_NUMBER=2;DB2_RETURNED_SQLCODE=-803;

RETURNED_SQLSTATE=23505;DB2_ROW_NUMBER=010;DB2_ERROR_CODE1=-110; DB2_ERROR_CODE2=13172739;DB2_ERROR_CODE4=13817814;DB2_SQLERRD1= -500;DB2_SQLERRD2=13172739;DB2_SQLERRD3=8;DB2_SQLERRD4=13817814;DB2_SQLERRD5=-490143744;DB2_INTERNAL_ERROR_POINTER=-110;DB2_MODULE_DETECTING_ERROR =DSNXRINS;MESSAGE_ID=DSN00803E ;SERVER_NAME=STLEC1;DB2_ORDINAL_TOKEN_1 =I1;DB2_ORDINAL_TOKEN_2 =000000020A;MESSAGE_TEXT=AN INSERTED OR UPDATED VALUE IS INVALID BECAUSE INDEX IN INDEX SPACE I1 CONSTRAINS COLUMNS OF THE TABLE SO NO TWO ROWS CAN CONTAIN DUPLICATE VALUES IN THOSE COLUMNS. RID OF EXISTING ROW IS X''000000020A'';CONDITION_NUMBER=3;DB2_RETURNED_SQLCODE=-803; RETURNED_SQLSTATE=23505;DB2_ROW_NUMBER=009;DB2_ERROR_CODE1=-110;DB2_ERROR_CODE2=13172739;DB2_ERROR_CODE4=13817814;

DB2_SQLERRD1=-500;DB2_SQLERRD2=13172739;DB2_SQLERRD3=8; DB2_SQLERRD4=13817814;DB2_SQLERRD5=-490143744;DB2_INTERNAL_ERROR_POINTER=-110; DB2_MODULE_DETECTING_ERROR=DSNXRINS;MESSAGE_ID=DSN00803E ;

SERVER_NAME=STLEC1;DB2_ORDINAL_TOKEN_1 =I1;DB2_ORDINAL_TOKEN_2 =000000020B;MESSAGE_TEXT=AN INSERTED OR UPDATED VALUE IS INVALID BECAUSE INDEX IN INDEX SPACE I1 CONSTRAINS COLUMNS OF THE TABLE SO NO TWO ROWS CAN CONTAIN DUPLICATE VALUES IN THOSE COLUMNS. RID OF EXISTING ROW IS X''000000020B'';’

This slide shows what you would really see if you issued the GET DIAGNOSTICS ALL statement. The data is returned in a VARCHAR(32672) string.

61



Summary

Host Variable ArraysMultiple-Row InsertMultiple-Row Fetch

DELETE WHERE CURRENT OFUPDATE WHERE CURRENT OF

GET DIAGNOSTICS

62

Faster FETCH & INSERTV8 Multiple Row Processing

Christopher J. Crone Senior Technical Staff Member/[email protected]

Session: C10Wednesday May 25th, 12:30 – 1:40 PM

Platform: DB2 UDB for z/OS

63

IBM Software Group


Reference Material

64



PL/1Example 2: You can retrieve 10 rows from the table CORPDATA.DEPARTMENT with:

DCL DEPTNO(10) CHAR(3),DEPTNAME(10) CHAR(29) VAR,MGRNO(10) CHAR(6),ADMRDEPT(10) CHAR(3);

DCL IND_ARRAY1(10)BIN FIXED(15);DCL IND_ARRAY2(10)BIN FIXED(15);...EXEC SQLDECLARE C1 CURSOR WITH ROWSET POSITIONING FORSELECT * FROM CORPDATA.DEPARTMENT;

...EXEC SQLFETCH FIRST ROWSET FORM C1 FOR 10 ROWS INTO

:DEPTNO :IND_ARRAY1,:DEPTNAME :IND_ARRAY2,:MGRNO :IND_ARRAY3,:ADMRDEPT :IND_ARRAY4;

In this example, several arrays of size 10 are declared for host variable values.Also, indicator arrays are declared to accept indicator values.Finally 10 rows are fetched into the host-variable-arrays and indicator-arrays.

65



C++Example 3: Declare an integer and varying character array to holdcolumns retrieved from a multi-row fetch statement

long serial_num(10);struct {

short len;char data [18];}name [10];

...EXEC SQLDECLARE C1 CURSOR FOR SELECT NAME, SERIAL#FROM CORPDATA.EMPLOYEE WITH ROWSET POSITIONING;

...EXEC SQL OPEN C1;EXEC SQLFETCH FIRST ROWSET FORM C1 FOR 10 ROWS INTO :NAME,:SERIAL_NUM;

In this example, we declare two host-variable-arrays serial_num, and name (which is a C/C++ structure).We then declare a cursor and FETCH 10 rows into the host-variable-arrays.

66



OPEN/ALLOCATE CURSOR

Users can use GET DIAGNOSTICS after OPEN or ALLOCATE to determine if a cursor is enabled for ROWSET positioning

ƒ DB2_SQL_ATTR_CURSOR_ROWSETN indicates that this cursor only supports row positioned operationsY indicates that this cursor supports rowset positioned operations

After OPEN or ALLOCATE CURSOR, new information is returned via GET DIAGNOSTICS that indicates the rowset capabilities of the cursor.

67



GET DIAGNOSTICS Syntax

Here is the first portion of the syntax diagram for the new GET DIAGNOSTICS statement.

68



GET DIAGNOSTICS Syntax (cont)

Condition information is retrieved by asking for information in the condition-information-item-name clause of GET DIAGNOSTICS.

69



GET DIAGNOSTICS Syntax (cont)

Information about the connection (local or remote) is returned using the connection-information-item-name clause.ALL indicates all diagnostic items for the last statement /condition/connection should be combined in one string. The data is returned as a string.

platform: db2 udb for z/os faster fetch & insert v8

Documents