sql stored procedures @ unms-nvsm 28-11-2013 - nvsm - sqlpl stor... · sql stored procedures @...

SQL Stored

Procedures

@

socmut.be

SQL Stored

Procedures

@

mutsoc.be

SQL Stored

Procedures

@

socmut.be

SQL Stored

Procedures

@

mutsoc.be

NVSM - Inf.Dir.Inf. - System MF - GSE DB2, Leuven

28/11/2013

AGENDA

About us

Stored

Procedures

Programming

The Good,

The Bad, …


28/11/2013

AGENDA

NVSM - Inf.Dir.Inf. - System MF - Page

Our Business

Our Infrastructure

Our Applications

Our Challenges

About us

Stored

Procedures

Programming

The Good,

The Bad, …

GSE DB2, Leuven

28/11/2013

Our business

• Health insurance and social

services


28/11/2013

Our business: some statistics

• Socialistische Mutualiteiten

– 11 health funds (federations)

– 1 Landsbond / Union

– 600 branches

– Socio-cultural organizations

• UNMS / NVSM

– Union Nationale des Mutualités Socialistes

– Nationaal Verbond der Socialistische Mutualiteiten

– 644 employees / 176 in IT / 8 system engineers mainframe

• Centralized services and administration

– For most of the federations

– Single point of contact for other social security organizations

• Market share: 28,17% (2011)

– 3.049.245 members (2011)


28/11/2013

Our mainframe infrastructure

• Hardware

– Dual site: Active / Passive

– 2 x z114

– 2 x DS8100 (PPRC)

– 2 x 3584 Tape Library

– Interconnecting DWDM

• Application software

– PL/1 V4.3

– IBM Datastudio V3.1.1

– Rational Developer for System z V8.5

– COGNOS BI V10

• System software

– z/OS V1.13

– CICS TS V4.1

– DB2 z/OS V10

– DB2 Connect 10.1

– IBM DB2 Cloning Tool

• Management software

– RMF

– CA Insight DB2

– CA Sysview

– CA RC/Migrator

– IBM Tivoli Decision Support

NVSM - Inf.Dir.Inf. - System MF -

GSE DB2, Leuven

28/11/2013

Our application environment

• 2 LPARs , 1 sysplex

– 3 user environments (T,A,P) + 2 system environments (D, I)

+ 1 sandbox per developer

– all 2-way data-sharing

• Applications

– Online applications • Terminal/GUI/Services

• CICS (PL/1)

• .NET (PL/1 and SQL/PL)

– Batch applications (PL/1)

• DB2 Production

– 66 databases / schema’s • 962GB compressed

• 1.884GB uncompressed


GSE DB2, Leuven

28/11/2013

Our current DB2 challenges

• Keep up …

– With migrating offline to online transactions

– With new projects (e-Health, …)

– With new versions and maintenance

– With our “agile” developers

– And other opportunities

• Planning DB2 V11 2014-Q4

GSE DB2, Leuven

28/11/2013 NVSM - Inf.Dir.Inf. - System MF -

AGENDA


Background

External

Native

About us

Stored

Procedures

Programming

The Good,

The Bad, …

GSE DB2, Leuven

28/11/2013

Background: why stored procedures?

• Early adopter (2005, DB2 V6)

– Modernization of user interface • Choice for .NET front-end

– Stored procedure as “application integration protocol” • From C# and DB2 Client via DB2 Connect and DDF

• Easy to integrate in client code

• Static SQL

• Traditional programming and deployment

– Can be written in PL/1

– Can wrap around existing PL/1 code

– Not very different from traditional routines

• Security

– GRANT EXECUTE ON PROCEDURE


28/11/2013

Background: workload and performance

GSE DB2, Leuven


?

TIP: Put sufficient intelligence in a

stored procedure


GSE DB2, Leuven


April 1

DB2

V10CM9

? ? ?

Workload activity

DDF service class


GSE DB2, Leuven


Workload activity DDF

PROD reporting classes

March 1

z10 z114

! ?

External stored procedures

GSE DB2, Leuven


• Written in PL/1

• External source & load module

• Definition in DB2 Catalog

create procedure SYSPROC.TBV8830 ( in vlog_header varchar(200) , out return_code char(5) , out sql_code integer , in p_fede smallint, in p_vzw char(5) , in p_datv date, in p_datt date ) language pli reads sql data dynamic result sets 1 external name TBV8830;

NOTE: External stored procedures can be written in many languages (C,Cobol,…) and also in the SQL

Procedural Language which we discuss later. External SQL stored procedures require a C compiler to

create a load module. This is not an option in our environment.

Native stored procedures

• Written in PL/1

• External source & load module

• Definition in DB2 Catalog

• Written in SQL language

• No load module

• Source is in the DB2 definition

GSE DB2, Leuven


create procedure SYSPROC.TBV8830 ( in vlog_header varchar(200) , out return_code char(5) , out sql_code integer , in p_fede smallint, in p_vzw char(5) , in p_datv date, in p_datt date ) language pli reads sql data dynamic result sets 1 external name TBV8830;

create procedure SYSPROC.TBV8930 ( in vlog_header varchar(200) , out return_code char(5) , out sql_code integer , in p_fede smallint, in p_vzw char(5) , in p_datv date, in p_datt date ) language sql reads sql data dynamic result sets 1 MAIN: begin ... END MAIN

Native stored procedures: why ?

• Lack of motivated PL/1 developers to “keep up”

– The Dinosaur prejudice

– Growing .NET developer team

• .NET developers want …

– … more than just do presentation logic

– … to increase their productivity

Code both presentation and data access logic

• Looking for more zIIP eligible workload

• Save CPU cost

• Versioning support

– We use V1 only (no versioning in PL/1 either)

• Better code portability

– But we have no porting plans

GSE DB2, Leuven


Future

potential

External vs Native stored procedures

• External SP’s: robust scalable infrastructure

GSE DB2, Leuven


DB2 DBM1

Stored Procedure Package

WLM Region

PL/1 Stored Procedure

Load module

DB2 DIST

DDF

Enclave

SRB TCB

External vs Native stored procedures

• Native SP’s: lightweight infrastructure

GSE DB2, Leuven


DB2 DBM1

SQL Stored Procedure Package

(includes logic)

DB2 DIST

DDF

Enclave

SRB

No task

switching

Code may

have longer

pathlength

savings

spendings Enclave

SRB

AGENDA


SQL/PL

Source management

Development tools

About us

Stored

Procedures

Programming

The Good,

The Bad, …

GSE DB2, Leuven

28/11/2013

What is SQL/PL

• SQL Procedural Language

– External SQL procedures

– Native SQL procedures

– Triggers

– User-defined functions

• Only has SQL statements

– And some SQL control statements (while, if, loop, goto)

• Runs inside the DB2 data management server (DBM1)

– SQL PL Engine integrated into RDBMS

GSE DB2, Leuven


An example

create procedure TBV893U (

in vlog_header varchar(200) ,

out return_code char(5) ,

out sql_code integer ,

in p_fede smallint,

in p_vzw char(5) ,

in p_datv date,

in p_datt date

)

language SQL

-- reads SQL data

modifies SQL data

asutime limit 2000

commit on return no

-- dynamic result sets 1

dynamic result sets 2

MAIN: begin

declare user char(8); declare sqlstate char(5); declare sqlcode integer; declare debug integer default null;

declare sysprint cursor with return for select text from sysibm.sysprint order by seqno asc; TEST_VZW1: begin --***************************************** --Test if VZW is valid numeric value --***************************************** declare minus420 condition for sqlstate '22018'; declare continue handler for minus420 begin -- This is condition where char does not -- convert to numeric set sql_code = sqlcode; set return_code = 'C001'; end; -- ------------------------------------------- -- if P_VZW is not a valid number, it sets the -- above minus420 (sqlcode-420) condition -- ------------------------------------------- if (p_vzw = 'ALL') then goto CONTROLE_NOK; elseif cast(p_vzw as decimal(5,0))<0 then set return_code = 'C002'; end if; end TEST_VZW1;

GSE DB2, Leuven


An example (continued)

TEST_VZW2: begin

--***************************************

--Test if VZW is valid business value

--***************************************

declare descr char(20);

if (return_code<>'00000') then

goto CONTROLE_NOK;

end if;

select sc0102 into test_vzw2.descr

from v00e0001 where as0201 = p_fede;

if (sqlcode<>0) then

set sql_code = sqlcode;

set return_code = 'C002';

end if;

end TEST_VZW2;

set debug = putskip(debug,’INPUT OK…’);

-- *******************

CONTROLE_NOK: begin

-- *******************

if (return_code<>'00000') then

goto BYE_BYE;

end if;

end CONTROLE_NOK;

-- *******************

CONTROLE_OK: begin

-- *******************

declare c1 cursor with return for

select … ;

declare c2 cursor with return for

select … ;

if (p_vzw = 'ALL') then

open c1;


else

open c2;


end if;

end CONTROLE_OK;

-- ********

BYE_BYE: begin

-- ********

end BYE_BYE;

set debug = putskip(debug,’All done !’);

if v_skips>0 then open sysprint; end if;

END MAIN#

GSE DB2, Leuven


TIP: Find a naming convention to

distinguish column names from variable

names in SQL/PL

TIP: You can exploit SYSIBM.SYSPRINT

for debugging purposes

An example (for the interested reader)

GSE DB2, Leuven


create function putskip (p_count integer, p_message varchar(4096))

returns integer deterministic no external action modifies sql data

begin

declare v_recnr integer default 0;

declare v_index integer default 1;

declare v_part varchar(254) ;

declare v_last integer default 0;

if p_count is not null then set v_recnr = p_count;

end if;

if p_message is not null then set v_last = length(p_message);

end if;

while v_index <= v_last do

set v_recnr = v_recnr + 1;

if v_index + 254 >= v_last then

set v_part = substr(p_message,v_index,254);

else

set v_part = substr(p_message,v_index) ;

end if;

insert into sysibm.sysprint values(v_recnr, v_part);

set v_index = v_index + length(v_part);

end while;

return v_recnr;

end#

Source management: PL/1 stored procedures

• 4 objects that should be in sync:

– Program source code • Source code library

• Source moves with deployment (Test Accp Prod)

– Load modules • Load module library

• Recreated in each environment (Test, Accp, Prod)

– DB2 package • DB2 Directory

• Recreated in each environment (Test, Accp, Prod)

– Stored procedure DDL • DDL library (& DB2 catalog)

• Source recreated in each environment (Test, Accp, Prod)

• Responsabilities

– Developer • Can independently change & test program, and deploy

• Cannot define the stored procedures (Test, Accp, Prod)

– Operations (TWS) • Deploys (automated) the program & package (Test Accp Prod)

– DBA • Deploys (manual) the stored procedure (Test, Accp, Prod)

• Query tuning review (Accp)

GSE DB2, Leuven


Conditions for SQL/PL

Stored Procedures

1. Sources must be in a

library

2. Developer must be

able to change and

test independently

3. Automation must be

possible for the

deployment

Source management: SQL/PL stored procedures

• Just 2 objects

– Source DDL statement: contains program logic

– DB2 package: automatically created at CREATE

• Developer

– Can CREATE and ALTER procedures in his own SCHEMA

– Can setup tables etc. in his own SCHEMA (a.k.a. SANDBOX)

– Must use &SYSUID..STORPROC.PDS as the source library

– Member name is 7 chars nnn893x

– Must prepare DDL for deployment to Test as member nnn893xT

• DBA

– Verifies and corrects the CREATE statement in nnn893xT

– Deploys in each environment (Test, Accp, Prod)

– Query tuning review (Accp)

• Operations

– No automation yet (neither for CREATE of PL/1 SP)

GSE DB2, Leuven


Source management: SQL/PL stored procedures

• Just 2 objects

– Source DDL statement: contains program logic

– DB2 package: automatically created at CREATE

• Developer

– Can CREATE and ALTER procedures in his own SCHEMA

– Can setup tables etc. in his own SCHEMA (a.k.a. SANDBOX)

– Must use &SYSUID..STORPROC.PDS as the source library

– Member name is 7 chars nnn893x

– Must prepare DDL for deployment to Test as member nnn893xT

• DBA

– Verifies and corrects the CREATE statement in nnn893xT

– Deploys in each environment (Test, Accp, Prod)

– Query tuning approval (Accp)

• Operations

– No automation yet (neither for CREATE of PL/1 SP)

GSE DB2, Leuven


TIP: Set a standard for SQL stored

procedure options, coding templates,

and provide an auto-correct tool

Development tools

• SPUFI

– Set your SQL Terminator different from “;”

GSE DB2, Leuven


Development tools

• Rational Developer for System z

GSE DB2, Leuven


Development tools

• FIXSQLPL

– Home produce to format code to (some of the) standards

– Verifies source member naming convention

– Verifies MAIN as label for the program

– Enforces CREATE options • LANGUAGE SQL

• QUALIFIER

• OWNER

• DYNAMIC RESULT SETS

• ASUTIME

• COMMIT ON RETURN NO

– Enforces mandatory EXIT HANDLER • Only 1 EXIT HANDLER, and we enforce it to catch & log exceptions

– Verifies mandatory signature • IN VLOG_HEADER VARCHAR(200)

• OUT RETURN_CODE CHAR(05)

• OUT SQL_CODE INTEGER

• Only 1 output parameter

– Some other minor verifications

– Converts from Sandbox-syntax to Test-syntax for deployment

GSE DB2, Leuven


AGENDA


The Good

The Bad

Our tips…

About us

Stored

Procedures

Programming

The Good,

The Bad, …

GSE DB2, Leuven

28/11/2013

The Good (1)

• Easy deployment

– No compile / link / bind

– DROP / CREATE

– Only 1 object to manage

– Suites both development via ISPF and graphical IDE

– “Agile”

• You may also need

– ALTER PROCEDURE REGENERATE ACTIVE VERSION • Kind of REBIND, but also takes SQL control statements into account

• Usually APAR HOLD action

– BIND PACKAGE(remote_location.collid) DEPLOY(local.package) • To leave source in one environment and create the “executable” in another

GSE DB2, Leuven


The Good (2)

• Successful roll-out in new projects

GSE DB2, Leuven


SYSSTAT

The Good (3)

• Unified Debugger in IBM DataStudio

– Add options on CREATE PROCEDURE

ALLOW DEBUG MODE

WLM ENVIRONMENT FOR DEBUG MODE your_wlm_environment

– Needs DB2-supplied stored procedures

See installation job DSNTIJSD

– Debug users must be granted permission

GRANT DEBUGSESSION

– In IBM DataStudio, go to Window>Preferences>Run/Debug>Routine Debugger>IBM and

choose the Session Manager option (we use built-in, but can also run

on host)

– May need to adjust ASUTIME on your stored procedure

GSE DB2, Leuven


The Good (4)

• CPU reduction? (1)

• It depends…

• Our observations are as follows

– If processing logic is trivial, then SQL/PL wins

– If more complex logic, and best practices are applied, then PL/1 wins

• Our conclusion

– CPU cost behaves sufficiently well to continue with SQL/PL

• To illustrate:

– Basic processing logic • Declare a cursor where result depends on input parameter

• Either – Return a result set with just opening the cursor (trivial processing)

– Return an output parameter by looping through the cursor (non-trivial processing)

GSE DB2, Leuven


The Good (4)

• CPU reduction? (2)

GSE DB2, Leuven


The Good (5)

• zIIP eligibility (1)

– Note that also a portion of PL/1 stored procedures is zIIP-eligible

GSE DB2, Leuven


The Good (5)


GSE DB2, Leuven


!!!

The Good (5)


GSE DB2, Leuven


The Bad (1)

• SQLCODE testing: prone to loops !

GSE DB2, Leuven


19/3/2013: Developer creates a

program loop in TEST

TIP: Enforce ASUTIME in TEST and set

your WLM definitions

The Bad (1)

• SQLCODE testing: prone to loops !

• Example:

GSE DB2, Leuven


declare v_name char(8) ; declare c1 cursor for select name from sysibm.sysdatabase fetch first 100 rows only with ur ; open c1; set p_sqlcode = 0; set p_result = ''; while p_sqlcode=0 do fetch c1 into v_name; if p_sqlcode<>0 then set p_rc = 'DONE!'; set p_sqlcode = sqlcode; else set p_result = p_result concat '-' concat v_name; set p_sqlcode = sqlcode; end if; end while;

declare v_name char(8) ; declare c1 cursor for select name from sysibm.sysdatabase fetch first 100 rows only with ur ; open c1; set p_sqlcode = 0; set p_result = ''; while p_sqlcode=0 do fetch c1 into v_name; set p_sqlcode = sqlcode; if p_sqlcode<>0 then set p_rc = 'DONE!'; else set p_result = p_result concat '-' concat v_name; set p_sqlcode = sqlcode; end if; end while;

The Bad (2)


– Not if called from CICS or Batch

GSE DB2, Leuven


DB2 DBM1

SQL Stored Procedure Package

(includes logic)

DB2 DIST

DDF

Enclave

SRB

Enclave

SRB

Address Space

Call SP

TCB TCB

zIIP-eligible

not zIIP-eligible

The Bad (2)


– See the bigger

picture

– Stored

procedures may

not be the major

zIIP opportunity

GSE DB2, Leuven


The Bad (3)

• Navigation by ‘GOTO’

• No support for copybooks

No pl/1 %include-like equivalent

• No support for routines

No support for existing PL/1 routines !

• No support for array variables

Arrays are supported from V11 ! Can’t wait !

• Each statement is an SQL statement

SQLCODE value changes at each statement

GSE DB2, Leuven


Our tips…

• Put sufficient intelligence in a stored procedure

• Find a naming convention to distinguish column names from variable names in SQL/PL

• You can exploit SYSIBM.SYSPRINT or equivalent for debugging purposes

• Set a standard for SQL stored procedure options, coding templates, and provide an auto-correct tool

• Enforce ASUTIME also in TEST and set your WLM definitions

• Distinguish your SQL/PL stored procedure performance records from PL/1 stored procedures

– RMF Workload Activity (SMF Type 72) records WLM service/reporting classes

– DB2 Accounting records Primary Authid / Correlation ID

GSE DB2, Leuven


[email protected]

[email protected]

We welcome your questions

!

Stijn Van den Bosch

Bart Verboven

THANK YOU !

GSE DB2, Leuven


mailto:[email protected]?subject=GSE Meeting Leuven 28/11/2013

mailto:[email protected]?subject=GSE Meeting Leuven 28/11/2013

sql stored procedures @ unms-nvsm 28-11-2013 - nvsm - sqlpl stor... · sql stored procedures @...

Documents