advanced and complex sql coding - gateway/400...1 iseries application development advanced and...

1

iSeries

Application Development

Advanced and

Complex SQL Coding

Sheryl M. Larsen

Sheryl M. Larsen, Inc.

May 12, 2011

Platform:

Gateway 400 User Group, St. Louis, MO

2 © Sheryl M. Larsen, Inc. 2000-2009

Contact: [email protected] (630) 399-3330 WWW.SMLSQL.COM

Sheryl M. Larsen, Inc. DB2 SQL Consulting & Education

Sheryl has 25 years experience

in DB2, has published articles,

white papers, webtv, animated

access paths:

Google Sheryl Larsen @

Bmc.com Ca.com Ibm.com

Sheryl Larsen is an internationally

recognized researcher, consultant

and lecturer, specializing in DB2

and is known for her extensive

expertise in SQL. She co-authored

a book, DB2 Answers, Osborne-

McGraw-Hill, 1999.

She was voted an inaugural IBM

Information Champion 2009 ,

IDUG Speaker Hall of Fame and, a

member of IBM’s DB2 Gold

Consultants program since 1994.

Currently, she is President of the

Midwest Database Users Group

(mwdug.org), and owns Sheryl M.

Larsen, Inc., a firm specializing in

Advanced DB2 Consulting and

Education.


Joins

Subqueries

Outer Joins

Common Table Expressions

Materialized Query Tables

EXCEPT

INTERCEPT

Browsing Result Sets

Table of Contents

SQL Skill Self Assessment Level Assessment = YOU CAN FULLY UNDERSTAND THE FEATURE AND PROPER USE OF:

0 You think SQL is a new energy drink

1 Simple SELECT statements, WITH clause, ORDER BY

2

WHERE clauses, BETWEEN, LIKE, IN(list), =, >=, >, <, <=, <>, NOT IN(list),

NOT LIKE, NOT BETWEEN, DISTINCT

3

Table joins (inner, outer, full), UNION, UNION ALL, CONCAT, static CURSORs,

FOR UPDATE OF, ROW_NUMBER, EXCEPT, INTERCEPT

4

noncorrelated and correlated subqueries, EXISTS, NOT EXISTS, FETCH

FIRST x ROWS ONLY, OPTIMIZE FOR x ROWS, MERGE, TRUNCATE

5

Indexable, Stage1 and Stage 2 predicate evaluation, multirow

FETCH/INSERT, GET DIAGNOSTICS, Scalar full SELECT

6

Table expressions/ common table expressions, GROUP BY, HAVING, IS NOT

DISTINCT FROM, embedded dynamic SQL, ORDER BY/FETCH FIRST in

subselect, FETCH CONTINUE, EXCEPT/INTERCEPT

7

CASE expressions, Global Temporary Table (GTT), Declared Temporary

Table (DTT), Dynamic Scrollable cursors, SEQUENCES/IDENTITY

8

Queries involving > 10 tables, INSERT within (SELECT , UPDATE, DELETE,

MERGE), Star Schema, GROUP BY expression, Browsing result sets

9

MQT (Materialized Query Tables), Recursive SQL, UNION in Views,

> 30 useful Built-in Functions, DENSE_RANK, RANK

10

Codes effective and efficient SQL applying performance rules and knows

when to use each SQL feature appropriately

Name: __________________

Before:_______

After:________


225 Tables in SQL Statement Limit

225 directly or indirectly identifying base table references:

SELECT …….. FROM T1, T2, ….T200 WHERE C1 IN

(SELECT ….. FROM T300 more tables

WHERE )

UNION ALL

(SELECT …… FROM T400 more tables

WHERE)

ORDER BY

FETCH FIRST 22 ROWS ONLY

WITH UR;

Full

SE

LE

CT

Subselect

Subquery


Inner Join

SELECT T42.PLCY_ACCT_NUM

,T52.SERV_ASGN_START_DT

FROM CLIENT T42

,AGENT T52

WHERE T42.PLCY_ID = T52.PLCY_ID

AND T42.LOB = ‘S’

AND T42.CLNT_PLCY_END_DT IS NULL

AND T52.SERV_ASGN_END_DT IS NULL

•List Most Restrictive Condition first within predicate type

•= , IN

•BETWEEN, >=, >,<,<=

•LIKE,

•Noncorrelated Subqueries

•Correlated Subqueries

•Produce a list of the accounts numbers, current servicing agent start date

for all active Insurance Payment Plan accounts.


Self Join Needed for Intra-Table Column Comparisons

» WHERE A.COL2 = A.COL5

» On different rows (accounts, policies, items, orders,

other unique ids, etc.)

» What Students Have the Same Classes?

SELECT SSS.SID, SSS.CLASS_ID, SSS.TIME_ID FROM SMLU_STUDENT_SCHED SS ,SMLU_STUDENT_SCHED SSS WHERE SSS.CLASS_ID = SS.CLASS_ID AND SSS.TIME_ID = SS.TIME_ID AND SSS.SID <> SS.SID AND SSS.SID NOT IN(list challenged student IDs) AND SS.SID = :challenged-sid

Self Join to get same

class/time rows Point to

Different

Rows

Homework

Challenged


Subqueries over Joins

Needed for Intra-Table Column Comparisons

» What Students Have at least one of the same Classes

as a given student?

SELECT SS.SID FROM SMLU_STUDENT_SCHED SS WHERE SS.SID NOT IN(list challenged student IDs)

AND EXISTS

(SELECT ‘TRUE OR FALSE’ FROM SMLU_STUDENT_SCHED SSS WHERE SSS.CLASS_ID = SS.CLASS_ID AND SSS.TIME_ID = SS.TIME_ID AND SSS.SID <> SS.SID AND SSS.SID = :challenged-sid)

Homework

Challenged


Left/Right Outer Joins

Get all of Client ids for single males

» Regardless if there are any email addresses

SELECT T40.CLNT_ID,

COALESCE(T71.EML_ADDR, 'NONE AVAIL')

FROM CLIENT T40

LEFT OUTER JOIN

EMAIL T71

ON T40.CLNT_ID = T71.CLNT_ID

WHERE T40.MRTL_STTS = 'S'

AND T40.GNDR = 'M'

Client

T40

Email

T71

SELECT Columns

FROM TABX X

LEFT JOIN TABY Y

ON X.COL1 = Y.COL1


Inner Join versus Left Join

Inner Join

TABX

Exceptions

TABY

Exceptions

SELECT Columns

FROM TABX X, TABY Y

WHERE X.COL1 = Y.COL1

Nested Common Table Expressions WITH COMMON1 (column list) AS

(SELECT PART, SUBPART, 1

FROM LEGO

WHERE PART = ‘L1’)

, COMMON2 (column list) AS

SELECT L.PART, L.SUBPART, N+1

FROM COMMON1 T1, LEGO L

WHERE T1.SUBPART = L.PART

AND N < 10 )

, COMMON3 (column list) AS

SELECT L.PART, L.SUBPART, N+1

FROM COMMON2 T1, BUILDING_BLOCKS L

WHERE T1.SUBPART = L.PART

AND N < 10 )

SELECT * FROM COMMON1

, TODDLERS2

WHERE …………..;



Materialized Query Tables Automatic Summary Tables (ASTs) are a subset

of MQTs

Can be ANY valid SELECT statement

CREATE TABLE MQT1 AS

SELECT dozens of aggregates

seven table join

nine column GROUP BY…

Or use your imagination .. ;


MQTs

Use them as “multi-table” indexes with or without aggregates

Optimizer is aware of all MQTs (for dynamically bound only for z/OS!) or direct reference for static

Auto query rewrite of base table queries to use MQTs if:

» CURRENT REFRESH AGE = „ANY‟

» CURRENT MAINTINED TABLE TYPES = „SYSTEM‟, „USER‟, „ALL‟

» ENABLE QUERY OPTIMIZATION is used

» Referential constraints between base tables exists

Matters more than you think…


…. To Build or not to Build

Generic MQT strategies

»HUGE MQT with many indexes

»Many small MQTs with few indexes

Optimal MQT strategy

»Most popular queries as MQTs • Common tables

• Common predicate sets

• Common data translations

»Indexes on the popular ones to allow access to subset of MQT


Popular Table Pivoting Syntax SELECT

SUM(CASE WHEN MONTH(SALES_DT) = 1 THEN AMOUNT END) AS JAN_AMT , SUM(CASE WHEN MONTH(SALES_DT) = 2 THEN AMOUNT END) AS FEB_AMT , SUM(CASE WHEN MONTH(SALES_DT) = 3 THEN AMOUNT END) AS MARCH_AMT , SUM(CASE WHEN MONTH(SALES_DT) = 4 THEN AMOUNT END) AS APRIL_AMT , SUM(CASE WHEN MONTH(SALES_DT) = 5 THEN AMOUNT END) AS MAY_AMT FROM SALES WHERE CUST_ID = :hv

JAN_AMT FEB_AMT MARCH_AMT APRIL_AMT MAY_AMT

238,50. 392.39 8,029.39 2,380.83 3,838.22

Generated

at

Execution


MQT

MQT INDEX:

CUST_ID

CREATE TABLE CUST_PIVOT AS (

SELECT CUST_ID

, SUM(CASE WHEN MONTH(SALES_DT) = 1 THEN AMOUNT END)

AS JAN_AMT


AS FEB_AMT


AS MARCH_AMT


AS APRIL_AMT


AS MAY_AMT

FROM SALES

GROUP BY CUST_ID)

DATA INITIALLY DEFERRED

REFRESH DEFERRED

MAINTAINED BY SYSTEM

ENABLE QUERY OPTMIZATION;


Query Speed Up

CUST_ID JAN_AMT FEB_AMT MARCH_AMT APRIL_AMT MAY_AMT

11 238,50. 342.39 1,029.39 5,380.83 3,438.22

12 298,50. 352.30 729.39 2,580.22 9,838.96

13 738,50. 362.32 8,769.39 7,360.33 538.48

14 1,038,50. 372.31 2,029.39 2,387.55 3,825.00

SELECT CUST_ID

, SUM(CASE WHEN MONTH(SALES_DT) = 1 THEN AMOUNT END) AS JAN_AMT

, SUM(CASE WHEN MONTH(SALES_DT) = 2 THEN AMOUNT END)AS FEB_AMT

, SUM(CASE WHEN MONTH(SALES_DT) = 3 THEN AMOUNT END)AS MARCH_AMT

, SUM(CASE WHEN MONTH(SALES_DT) = 4 THEN AMOUNT END)AS APRIL_AMT

, SUM(CASE WHEN MONTH(SALES_DT) = 5 THEN AMOUNT END)AS MAY_AMT

FROM SALES

WHERE CUST_ID = 12

Accessed

at

Execution SELECT * FROM CUST_PIVOT

WHERE CUST_ID = 12

CUST_PIVOT


EXCEPT, INTERSECT NAME

Able

Able

Baker

Baker

R1

NAME

Able

Able

Charlie

Charlie

R2

SELECT

……

……

……

AS R1

UNION

SELECT

…….

…….

AS R2

NAME

Able

Baker

Charlie

SELECT

……

……

……

AS R1

UNION

ALL

SELECT

…….

…….

AS R2

NAME

Able

Able

Baker

Baker

Able

Able

Charlie

Charlie

SELECT

……

……

……

AS R1

EXCEPT

SELECT

…….

…….

AS R2

NAME

Baker

SELECT

……

……

……

AS R1

EXCEPT

ALL

SELECT

…….

…….

AS R2

NAME

Baker

Baker

SELECT

……

……

……

AS R1

INTERSECT

SELECT

…….

…….

AS R2

NAME

Able

SELECT

……

……

……

AS R1

INTERSECT

ALL

SELECT

…….

…….

AS R2

NAME

Able

Able

Able

Able

Baker Baker

Able Able

Able Able

Charlie Charlie

Find Things that Have ALL

List the clients that have at least one active policy in ALL lines of

business offered by Insurance Company.

SELECT DISTINCT PR1.CLNT_ID

FROM POLICY_ROLE PR1

WHERE NOT EXISTS (SELECT DISTINCT P1. LOB

FROM POLICY P1 )

EXCEPT

(SELECT DISTINCT PR2.LOB

FROM POLICY_ROLE PR2

WHERE

PR2.CLNT_ID = PR1.CLNT_ID

AND PR2.CLNT_PLCY_END_DT IS NULL )


A

F

H

L

- =

A

F

H

L

H

A

F

L

All

701 801

A

F

H

L

All

- =

False

True

EXCEPT Example If the employee works on every project located in Denver, then

list the employee‟s social security number and name.

SELECT NAME, SSN

FROM EMPLOYEE E

WHERE NOT EXISTS

((SELECT PROJECT.PNUMBER

FROM PROJECT

WHERE PLOCATION = „DENVER')

EXCEPT

(SELECT W.PNUMBER

FROM WORKSON

WHERE E.SSN = W.SSN));


P11

P26

P84

P99

All

Test one SSN at a time

If their list matches

They win!

21

z/OS


Repositioning

Cursors

Sheryl M. Larsen


Platform:


EMP Table and Cursor EMPNO EMP_NAME EMP_ADDR EMP_PHONE

1 JOE DOWNERS GROVE 5552833820

2 SAM ELMHURST 5553824857

4 JIM AURORA 5559382737

6 KELLY LIBERTYVILLE 5551293874

8 MIKE BARRINGTON 5551267480

DELCLARE EMPCSR CURSOR FOR SELECT EMP_NAME, EMP_ADDR, EMP_PHONE FROM EMP WHERE EMPNO > :lastempno ORDER BY EMPNO FETCH FIRST 2 ROWS ONLY;

© Sheryl M. Larsen, Inc. 2000-2009

23

Get First 2 Rows EMPNO EMP_NAME EMP_ADDR EMP_PHONE



4 JIM AURORA 5559382737



SET :lastempno = 0 , :page0 in COM AREA

OPEN EMPCSR;

FETCH EMPCSR FOR 2 ROWS;

Store :lastempno = :last-fetched-empno in COM AREA and :page1

Display screen


Get Next 2 Rows EMPNO EMP_NAME EMP_ADDR EMP_PHONE



4 JIM AURORA 5559382737



Get :lastempno from COM AREA (=2 for page1)

OPEN EMPCSR;

FETCH EMPCSR FOR 2 ROWS;

Store :lastempno = :last-fetched-empno in COM AREA and :page2


Go Back 1 Page EMPNO EMP_NAME EMP_ADDR EMP_PHONE



4 JIM AURORA 5559382737



Get :lastempno from COM AREA for Page 0 (=0)

OPEN EMPCSR;

FETCH EMPCSR;

Store :firstempno and :page2 in COM AREA

FETCH EMPCSR;

Store :lastempno = :last-fetched-empno in COM AREA

Always go an extra page back so the > works


BRWSUM Table

COL1 COL2 COL3 COL4 COL5

A 1 TT 99 Data ….

A 1 UU 77 Data ….

A 4 SS 66 Data ….

B 2 RR 66 Data ….

B 3 RR 77 Data ….

B 3 RR 88 Data ….

B 3 SS 66 Data ….

B 4 SS 99 Data ….

B 4 UU 88 Data ….

C 1 SS 66 Data ….

C 1 SS 77 Data ….

Makes row unique, should have index COL1.COL2.COL3.COL4


Declare Cursor


A 1 TT 99 Data ….

A 1 UU 77 Data ….

A 4 SS 66 Data ….

B 2 RR 66 Data ….

B 3 RR 77 Data ….

B 3 RR 88 Data ….

B 3 SS 66 Data ….

B 4 SS 99 Data ….

B 4 UU 88 Data ….

C 1 SS 66 Data ….

C 1 SS 77 Data ….

DELCLARE BRWSUM1 CURSOR FOR

SELECT COL5 FROM BRWSUM

WHERE ? AND ? AND ? AND ? AND ? AND ? AND ?

ORDER BY COL1, COL2, COL3, COL4

FETCH FIRST 5 ROWS ONLY;

What should these be?


Declare BRWSUM1Cursor


A 1 TT 99 Data ….

A 1 UU 77 Data ….

A 4 SS 66 Data ….

B 2 RR 66 Data ….

B 3 RR 77 Data ….

B 3 RR 88 Data ….

B 3 SS 66 Data ….

B 4 SS 99 Data ….

B 4 UU 88 Data ….

C 1 SS 66 Data ….

C 1 SS 77 Data ….

DELCLARE BRWSUM1 CURSOR FOR SELECT COL5

FROM BRWSUM WHERE ((COL1 = :col1-last AND COL2 = :col2-last AND COL3 = :col3-last AND COL4 > :col4-last) OR (COL1 = :col1-last AND COL2 = :col2-last AND COL3 > :col3-last) OR (COL1 = :col1-last AND COL2 > :col2-last) OR (COL1 > :col1-last))

ORDER BY COL1, COL2, COL3, COL4 FETCH FIRST 5 ROWS ONLY;


Get First 5 Rows

OPEN BRWSUM1;

FETCH BRWSUM1 FOR 5 ROWS;

Store :col1-4-last = :last-fetched-row in COM AREA and :page1

Display screen


A 1 TT 99 Data ….

A 1 UU 77 Data ….

A 4 SS 66 Data ….

B 2 RR 66 Data ….

B 3 RR 77 Data ….

B 3 RR 88 Data ….

B 3 SS 66 Data ….

B 4 SS 99 Data ….

B 4 UU 88 Data ….

C 1 SS 66 Data ….

C 1 SS 77 Data ….

SET

:col1-last = low values, :col2-last = 0, :col3-last = low values,:col4-last = 0,

:page0 in COM AREA


Get Next 5 Rows

OPEN BRWSUM1;

( (COL1 = ‘B’ AND COL2 = 3 AND COL3 = ‘RR’ AND COL4 > 77) OR (COL1 = ‘B’ AND COL2 = 3 AND COL3 > ‘RR’) OR

(COL1 = ‘B’ AND COL2 > 3) OR (COL1 > ‘B’))

FETCH BRWSUM1 FOR 5 ROWS;


Display screen


A 1 TT 99 Data ….

A 1 UU 77 Data ….

A 4 SS 66 Data ….

B 2 RR 66 Data ….

B 3 RR 77 Data ….

B 3 RR 88 Data ….

B 3 SS 66 Data ….

B 4 SS 99 Data ….

B 4 UU 88 Data ….

C 1 SS 66 Data ….

C 1 SS 77 Data ….

Get from COM AREA

:col1-last = „B‟, :col2-last = 3, :col3-last = „RR‟,:col4-last =77

First Screen


Go Back 1 Page

OPEN BRWSUM1;

( (COL1 = low AND COL2 = 0 AND COL3 = low AND COL4 > 0) OR (COL1 = low AND COL2 = 0 AND COL3 > low) OR (COL1 = low AND COL2 > 0) OR (COL1 > 0))

FETCH BRWSUM1;

Loop 5 times


Display screen


A 1 TT 99 Data ….

A 1 UU 77 Data ….

A 4 SS 66 Data ….

B 2 RR 66 Data ….

B 3 RR 77 Data ….

B 3 RR 88 Data ….

B 3 SS 66 Data ….

B 4 SS 99 Data ….

B 4 UU 88 Data ….

C 1 SS 66 Data ….

C 1 SS 77 Data ….

Get from COM AREA for page0

:col1-last = low values :col2-last = 0, :col3-last = low values,:col4-last =0

Last Screen

32

z/OS


Thank You!

Sheryl M. Larsen


www.smlsql.com

Platform:

iSeries Information Center, Version 5 Release 3

http://publib.boulder.ibm.com/infocenter/iseries/v5r3/index.jsp

http://publib.boulder.ibm.com/infocenter/iseries/v5r3/topic/rzahg/icmain.htm



advanced and complex sql coding - gateway/400...1 iseries application development advanced and...

Documents