large objects with db2 zos and os390

ibm.com/redbooks

Front cover

with DB2 for z/OS and OS/390

Paolo BruniPatric Becker

Michael DewertBob Riehle

Define LOBs, see how they work, and see how to store them

Manage LOBs in operational environments

Use LOBs in applications and DB2 Extenders

http://www.redbooks.ibm.com/


International Technical Support Organization

Large Objects with DB2 for z/OS and OS/390

June 2002

SG24-6571-00

© Copyright International Business Machines Corporation 2002. All rights reserved.Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBMCorp.

First Edition (June 2002)

This edition applies to Version 7 of IBM DATABASE 2 Universal Database Server for z/OS and OS/390 (DB2 UDB for z/OS and OS/390 Version 7), program number 5675-DB2.

Note: Before using this information and the product it supports, read the information in “Notices” on page xiii.

Contents

Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii

Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix

Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi

Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiiiTrademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiv

Summary of changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xvJune 2002, First Edition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xv

November 2004, First Update . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xv

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xviiThe team that wrote this redbook. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xviiBecome a published author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xviiiComments welcome. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix

Chapter 1. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1 Summary of considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

Chapter 2. Large objects with DB2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.1 Object orientation with DB2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62.2 Large objects with DB2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.3 The new data types. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2.3.1 LOB locators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92.3.2 The ROWID . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Chapter 3. Creating LOBs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133.1 Data conversion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143.2 Storing LOBs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

3.2.1 How to store them in DB2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163.2.2 Creating a table containing LOBs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193.2.3 Creating the LOB table spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223.2.4 Creating the auxiliary tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283.2.5 The auxiliary index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293.2.6 Adding a LOB column to an existing table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

3.3 The CURRENT RULES special register . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313.3.1 Impact on CREATE and DROP LOBs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313.3.2 Impact on cursors fetching LOB values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

3.4 Feeding a LOB column . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333.4.1 LOAD with LOB columns smaller than 32 KB. . . . . . . . . . . . . . . . . . . . . . . . . . . . 343.4.2 Inserting LOBs via host application. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

Chapter 4. Using LOBs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434.1 SQL semantics with LOBs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 444.2 LOB locators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

4.2.1 Getting to know LOB locators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 474.3 Handling BLOBs, CLOBs, DBCLOBs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

4.3.1 Manipulating a LOB without retrieving it . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

© Copyright IBM Corp. 2002 iii

4.3.2 Unloading a LOB via application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 544.3.3 Finding the nth occurrence of a string. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

4.4 LOBs are different DB2 objects. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 584.4.1 Storage of LOBs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

4.5 Locking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 624.5.1 Locks with simple reads . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 634.5.2 Using ISOLATION (UR) for LOBs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 644.5.3 Locks with INSERT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 654.5.4 Locks with DELETE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 654.5.5 Locks with UPDATE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 674.5.6 Some general information about locking. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

4.6 Details about ROWID . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

Chapter 5. DB2 Extenders and LOBs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 735.1 Generalities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 745.2 AVI Extenders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

5.2.1 Desktop. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 765.2.2 Database. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 765.2.3 WLM environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 765.2.4 Administration tasks to enable a table to use extenders. . . . . . . . . . . . . . . . . . . . 77

5.3 XML Extenders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 815.3.1 XML Toolkit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 835.3.2 WLM application environment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 835.3.3 General host requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 835.3.4 Workstation requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

5.4 XML and LOBs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

Chapter 6. Data administration with LOBs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 916.1 The DB2 Catalog and LOBs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

6.1.1 Some catalog definitions for LOBs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 926.1.2 LOBs defined in DB2 catalog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

6.2 Utilities. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 946.2.1 LOAD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 946.2.2 DSNTIAUL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 966.2.3 UNLOAD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 966.2.4 COPY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 976.2.5 REPORT TABLESPACESET . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 996.2.6 RUNSTATS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 996.2.7 REORG. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1006.2.8 CHECK LOB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1046.2.9 CHECK DATA. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1056.2.10 CHECK INDEX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1086.2.11 DSN1COPY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1086.2.12 REPAIR. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1096.2.13 RECOVERY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

6.3 Recovery scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1116.4 Operational scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117

Chapter 7. Performance with LOBs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1197.1 LOB materialization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120

7.1.1 Using data spaces for LOBs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1237.2 Buffer pools and group buffer pools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1257.3 LOBs performance considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127

7.3.1 Logging with LOBs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

iv Large Objects with DB2 for z/OS and OS/390

7.3.2 LOBs processing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1297.3.3 LOBs read performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1307.3.4 Comparing SQL accounting profiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1317.3.5 LOBs write performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1327.3.6 IFCIDs enhancements for LOBs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1337.3.7 LOBs recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136

Appendix A. Sample jobs output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137A.1 DDL for a LOB environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138A.2 Sample of DB2 PM accounting trace output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139A.3 Using file reference variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144

Appendix B. Unicode implementation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147B.1 Generate the conversion table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148B.2 Activate the UNICODE table. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149

Appendix C. Recent maintenance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151C.1 Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152C.2 LOB related maintenance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152

Appendix D. Additional material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155Locating the Web material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155Using the Web material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155

System requirements for downloading the Web material . . . . . . . . . . . . . . . . . . . . . . . 156How to use the Web material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156

Abbreviations and acronyms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157

Related publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161

Other resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161Referenced Web sites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162How to get IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162

IBM Redbooks collections. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163

Contents v

vi Large Objects with DB2 for z/OS and OS/390

Figures

2-1 Assigning a LOB locator for a LOB value . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102-2 LOB structure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123-1 Mixed client /server environment with different data types . . . . . . . . . . . . . . . . . . . . 143-2 Unicode considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153-3 Association between base table and auxiliary table . . . . . . . . . . . . . . . . . . . . . . . . . 173-4 Base table with two LOB columns and the two associated LOB table spaces . . . . . 173-5 Partitioned base table containing two LOB columns . . . . . . . . . . . . . . . . . . . . . . . . . 183-6 Association between base table, ROWID, auxiliary table, and LOB table space . . . 193-7 Catalog description for table BOOK_BASE_TABLE . . . . . . . . . . . . . . . . . . . . . . . . . 213-8 Applying changes to a LOB table space created with LOG NO. . . . . . . . . . . . . . . . . 243-9 SYSIBM.SYSCOLUMNS contents for TBNAME= BOOK_AUX_TABLE . . . . . . . . . . 293-10 Index keys for an auxiliary index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293-11 Secondary chain of locators. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393-12 Primary chain of locators containing secondary chains . . . . . . . . . . . . . . . . . . . . . . . 404-1 Concurrent LOB access using LOB locators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 484-2 Accessing a LOB without a locator reference using ISOLATION (CS) . . . . . . . . . . . 564-3 Processing a LOB using a locator reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 574-4 LOB value spanned over pages using chunks and non-chunks . . . . . . . . . . . . . . . . 604-5 LOB data pages chunked together via space map and LOB map pages . . . . . . . . . 614-6 Lock escalation on LOBS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 634-7 Scan lock sequence. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 644-8 Scan lock sequence using uncommitted read . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 644-9 Insert lock sequence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 654-10 Delete lock sequence. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 664-11 Locks acquired by mass delete . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 664-12 Lock sequence when updating a LOB column. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 675-1 Extender architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 745-2 XML columns with side tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 855-3 XML column. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 875-4 XML collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 886-1 Fragmented LOB table space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1026-2 Non-fragmented LOB table space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1036-3 Recovery environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1126-4 Recovering the auxiliary table with full image copy . . . . . . . . . . . . . . . . . . . . . . . . . 1136-5 Rebuilding the index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1146-6 Delete row from base table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1166-7 Take a full image copy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1177-1 DB2 materialization overview picture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1207-2 A SELECT application and LOB materialization . . . . . . . . . . . . . . . . . . . . . . . . . . . 1217-3 An INSERT application and LOB materialization. . . . . . . . . . . . . . . . . . . . . . . . . . . 1227-4 Panel DSNTIP7 showing storage LOB values. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1247-5 Separation of LOB buffer pools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1257-6 LOB group buffer pool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1267-7 Read performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1307-8 Write performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132

© Copyright IBM Corp. 2002 vii

viii Large Objects with DB2 for z/OS and OS/390

Tables

2-1 Typical average size for large objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83-1 Choosing a LOB page size that minimizes getpages . . . . . . . . . . . . . . . . . . . . . . . . 263-2 Choosing a LOB page size for LOBs all of the same size . . . . . . . . . . . . . . . . . . . . . 263-3 Primary and secondary quantity with LOBs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263-4 Summary of data set, partition, and partitioned table space sizes. . . . . . . . . . . . . . . 273-5 LOB table space created using CURRENT RULES STD . . . . . . . . . . . . . . . . . . . . . 323-6 Auxiliary index created using CURRENT RULES STD . . . . . . . . . . . . . . . . . . . . . . . 323-7 Where large objects come from and how . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334-1 Casting large objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 465-1 UDTs used with AVI Extenders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 785-2 DEFAULT UDTs after enabling the XML SERVER . . . . . . . . . . . . . . . . . . . . . . . . . . 815-3 XML Extender storage functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 815-4 Storage UDFs used with XML Extenders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 825-5 Extract UDFs used with XML Extenders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 825-6 User defined types. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 896-1 Impact of logging on LOB tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 956-2 SYSIBM.SYSLOBSTATS columns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1006-3 Allowed keywords and options. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1006-4 Resetting AUXW flags . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1076-5 Resetting AUX CHECK pending status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1076-6 Object status after different recovery scenarios. . . . . . . . . . . . . . . . . . . . . . . . . . . . 1107-1 Two subsystem parameters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1247-2 Current IFCID providing information regarding LOBs . . . . . . . . . . . . . . . . . . . . . . . 133C-1 Requirements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152C-2 DB2 V7 LOB related APARs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152

© Copyright IBM Corp. 2002 ix

x Large Objects with DB2 for z/OS and OS/390

Examples

2-1 Host variable definitions for LOB locators in COBOL . . . . . . . . . . . . . . . . . . . . . . . . 112-2 What the DB2 precompiler makes of LOB locators . . . . . . . . . . . . . . . . . . . . . . . . . . 113-1 DDL for a base table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203-2 DDL for a LOB table space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223-3 DDL for an auxiliary table. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283-4 DDL for an auxiliary table containing data of one base table partition. . . . . . . . . . . . 283-5 DDL for an auxiliary Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293-6 Displaying a database for LOBs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303-7 Adding a ROWID column. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303-8 Adding a LOB column . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313-9 LOAD a base table containing a LOB columns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343-10 Tieing rows together in one variable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353-11 host variable declaration for a LOB column. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353-12 What the DB2 precompiler makes of your host variable declaration . . . . . . . . . . . . . 353-13 Inserting a single LOB value using one host variable . . . . . . . . . . . . . . . . . . . . . . . . 363-14 Pseudo-code inserting LOBs > 32 KB with one locator chain . . . . . . . . . . . . . . . . . . 363-15 Pseudo-code inserting LOBs > 32 KB with multiple locator chains . . . . . . . . . . . . . . 404-1 Syntax for FREE locator and HOLD locator. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 494-2 Delete Chapter 8 of book CLOB using locators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 524-3 Updating a part of a CLOB. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 534-4 Inserting a text at a particular position . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 544-5 Unloading LOB data using one host variable. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 544-6 Unloading a LOB using locators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 554-7 Finding a specific occurrence of a string . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 574-8 DDL for a table containing a ROWID column . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 694-9 Inserting a row in CUSTOMER table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 694-10 ROWID value of a table created with ROWID column. . . . . . . . . . . . . . . . . . . . . . . . 694-11 DSN1PRNT of ROWID in hex value . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 694-12 ALTER TABLE adding a ROWID column . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 704-13 ROWID value of a table where a ROWID column was added . . . . . . . . . . . . . . . . . . 704-14 DSN1PRNT of ROWID in hex value after adding and updating a row . . . . . . . . . . . 704-15 ROWID values of updated and inserted columns . . . . . . . . . . . . . . . . . . . . . . . . . . . 704-16 DSN1PRNT of updated and inserted ROWIDs in hex value . . . . . . . . . . . . . . . . . . . 715-1 Sample DMBWLM1 procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 765-2 DMBENVAR DD card input . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 775-3 Example of ENABLE SERVER command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 775-4 Defining the table for an image . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 795-5 Example of ENABLE TABLE command. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 795-6 Example Enable Column Command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 795-7 Storing an image . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 805-8 Browsing an image . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 805-9 WLM procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 835-10 Retrieval of an XML column document . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 876-1 Select from SYSIBM.SYSCOLUMS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 926-2 Select from SYSIBM.SYSTABLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 936-3 DB2 catalog query . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 936-4 Output for Load job inserting into table with CLOB column . . . . . . . . . . . . . . . . . . . . 956-5 Output from Unload with LOB column < 32 KB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

© Copyright IBM Corp. 2002 xi

6-6 Output from DSNTIAUL where column length > 32 KB . . . . . . . . . . . . . . . . . . . . . . . 966-7 Using UNLOAD with truncation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 976-8 Using the UNLOAD utility on a base table with row size > 32 KB . . . . . . . . . . . . . . . 976-9 Copy of base and LOB table space in one statement . . . . . . . . . . . . . . . . . . . . . . . . 986-10 Query of SYSCOPY for common RBA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 986-11 REPORT TABLESPACESET with CURRENT RULES STD created LOBs . . . . . . . 996-12 Table SYSIBM.SYSLOBSTATS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1016-13 Trying to use REORG UNLOAD EXTERNAL on a base table row > 32 KB . . . . . . 1046-14 DSN1COPY error for LOB table space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1086-15 JCL for DSN1COPY OBIDXLAT for LOB table space. . . . . . . . . . . . . . . . . . . . . . . 1086-16 Display database after recovery, using last full image copy . . . . . . . . . . . . . . . . . . 1136-17 Display database after Rebuild Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1146-18 Check Data utility output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1156-19 Display database after Check Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1156-20 Display database at the end . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1167-1 MVS display command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1237-2 DB2 PM accounting miscellaneous values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1237-3 DB2 PM statistics miscellaneous values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1237-4 Calculation for LOB space maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1267-5 Calculation of group buffer pool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1277-6 Inserting a 1 MB LOB with LOG YES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1287-7 Select into host variable. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1317-8 Accounting trace output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131A-1 Enabled Extender tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144A-2 Use of file reference. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144B-1 JCL to create the conversion table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148B-2 Output from the generation job . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148B-3 Contents of CUNUNI01 in SYS1.PARMLIB. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149B-4 Activate the new UNICODE table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150B-5 UNI=ALL output in SYSLOG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150

xii Large Objects with DB2 for z/OS and OS/390

Notices

This information was developed for products and services offered in the U.S.A.

IBM may not offer the products, services, or features discussed in this document in other countries. Consult your local IBM representative for information on the products and services currently available in your area. Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any IBM intellectual property right may be used instead. However, it is the user's responsibility to evaluate and verify the operation of any non-IBM product, program, or service.

IBM may have patents or pending patent applications covering subject matter described in this document. The furnishing of this document does not give you any license to these patents. You can send license inquiries, in writing, to: IBM Director of Licensing, IBM Corporation, North Castle Drive Armonk, NY 10504-1785 U.S.A.

The following paragraph does not apply to the United Kingdom or any other country where such provisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of express or implied warranties in certain transactions, therefore, this statement may not apply to you.

This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice.

Any references in this information to non-IBM Web sites are provided for convenience only and do not in any manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the materials for this IBM product and use of those Web sites is at your own risk.

IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring any obligation to you.

Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products.

This information contains examples of data and reports used in daily business operations. To illustrate them as completely as possible, the examples include the names of individuals, companies, brands, and products. All of these names are fictitious and any similarity to the names and addresses used by an actual business enterprise is entirely coincidental.

COPYRIGHT LICENSE: This information contains sample application programs in source language, which illustrates programming techniques on various operating platforms. You may copy, modify, and distribute these sample programs in any form without payment to IBM, for the purposes of developing, using, marketing or distributing application programs conforming to the application programming interface for the operating platform for which the sample programs are written. These examples have not been thoroughly tested under all conditions. IBM, therefore, cannot guarantee or imply reliability, serviceability, or function of these programs. You may copy, modify, and distribute these sample programs in any form without payment to IBM for the purposes of developing, using, marketing, or distributing application programs conforming to IBM's application programming interfaces.

© Copyright IBM Corp. 2002. xiii

TrademarksThe following terms are trademarks of the International Business Machines Corporation in the United States, other countries, or both:

AIX®CICS®DB2®DB2 Connect™DB2 Extenders™DFS™DFSMS/MVS®DRDA®Enterprise Storage Server™e (logo)®

IBM®IMS™MVS™OS/390®Perform™QBIC®QMF™RACF®RAMAC®Redbooks (logo)™

Redbooks™S/390®Sequent®SP™System/390®VTAM®WebSphere®z/OS™

The following terms are trademarks of International Business Machines Corporation and Lotus Development Corporation in the United States, other countries, or both:

Lotus® Lotus Notes® Notes®

The following terms are trademarks of other companies:

ActionMedia, LANDesk, MMX, Pentium and ProShare are trademarks of Intel Corporation in the United States, other countries, or both.

Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both.

Java and all Java-based trademarks and logos are trademarks or registered trademarks of Sun Microsystems, Inc. in the United States, other countries, or both.

C-bus is a trademark of Corollary, Inc. in the United States, other countries, or both.

UNIX is a registered trademark of The Open Group in the United States and other countries.

SET, SET Secure Electronic Transaction, and the SET Logo are trademarks owned by SET Secure Electronic Transaction LLC.

Other company, product, and service names may be trademarks or service marks of others.

xiv Large Objects with DB2 for z/OS and OS/390

Summary of changes

This section describes the technical changes made in this edition of the book and in previous editions. This edition may also include minor corrections and editorial changes that are not identified.

Summary of Changesfor SG24-6571-00for Large Objects with DB2 for z/OS and OS/390as created or updated on November 9, 2004.

June 2002, First EditionThis revision reflects the addition, deletion, or modification of new and changed information described below.

November 2004, First Update

Changed informationChange bars identify the corrected areas.

� Corrected information on Reorg and Recover execution against LOBs in the text following 6.2.7, “REORG” on page 100 and removed the previously numbered 6-3 Figure.

© Copyright IBM Corp. 2002. All rights reserved. xv

xvi Large Objects with DB2 for z/OS and OS/390

Preface

The amount of data to be managed by workstations, middleware systems and mainframes, keeps growing day by day. The times are changing, and so the requirements for a database management system (DBMS) are changing.

With the incorporation of object-orientation into DB2 for z/OS and OS/390 (simply referred to as DB2 throughout this redbook), you can define new data types and functions. Some of the data objects you want to model may well be very large and complex. The foundation of object-relational extension, introduced with DB2 UDB Server for OS/390 Version 6 has two major functions. On one hand it has support for large objects (LOB); and, on the other hand it has support for user defined functions, user defined distinct types, and triggers. These large objects can contain pictures, images, text-documents or even movies, and can be stored directly in the DBMS with sizes up to 2 gigabytes per object and 4,000 terabytes per LOB column. The largest available size in DB2 before LOBs was 32 kilobytes, for data defined as a varchar.

Normally, large objects are used and manipulated through graphical user interfaces from a workstation. So with the implementation of LOBs we can exploit the functionalities of the DB2 family and the enterprise server capacity that DB2 for z/OS provides today. The introduction of these new data types implies some changes in the administration processes and programming techniques.

In this IBM Redbook we describe the new data types, and provide some useful information on how to design and implement LOBs. We also offer examples of their use, programming considerations, and the new processes which are necessary for administration and maintenance.

The team that wrote this redbookThis redbook was produced by a team of specialists from around the world working at the International Technical Support Organization, San Jose Center.

Paolo Bruni is a Certified Consultant IT Architect on assignment at the International Technical Support Organization, San Jose Center, where he conducts projects on all areas of Data Management. Before joining the ITSO in 1998, Paolo worked at IBM Italy. During his many years with IBM, in development and in the field, Paolo’s work has been principally related to database systems.

Patric Becker is an Application Programmer with Sparkassen Informatik GmbH and Co. KG in Germany. Since joining the company in 1997, Patric has been responsible for several, high availability, Customer Relationship Management, DB2 and IMS applications. For DB2, he has also assumed the role of a database administrator and he was involved in evaluating and applying the new functions of DB2 for OS/390 Version 6 and Version 7.

Michael Dewert is a Systems Programmer with more than 12 years experience with DB2 for OS/390. Before joining IBM Germany in 2001, Michael has worked for a consortium of Savings Banks in Northern Germany and his responsibilities included IMS and CICS as well as DB2. More recently he was involved in DB2 Quality Partnership Programs and his area of expertise has been expanded to include performance tools, data sharing, disaster recovery, and distributed platforms.

© Copyright IBM Corp. 2002 xvii

Bob Riehle is a Consultant with 10 years of experience in DB2, having previously worked with IMS for several years. His areas of special interest within DB2 include performance, applications, and subsystem tuning. Bob’s work with his customers ranges from systems high-level architecture and connectivity, to database design and performance, largely related to DB2.

Thanks to the following people for their contributions to this project:

Emma JacobsYvonne LyonBart SteegmansInternational Technical Support Organization, San Jose Center

Rich ConwayBob HaimowitzInternational Technical Support Organization, Poughkeepsie Center

Gabrielle VelezInternational Technical Support Organization, Rochester Center

Chuck BonnerKarelle CornwellChris CroneWillie FaveroSusan MalaikaRoger MillerAkira ShibamyiaKalpana ShyamDave WangJane WangJay YothersGary WilmotIBM Silicon Valley Laboratory, San Jose

Ian CookSarah EllisIBM PIC, Hursley, UK

Bernhard BaumgartnerIBM Switzerland

Toine MichielseIndependent Consultant, The Netherlands

Become a published authorJoin us for a two- to seven-week residency program! Help write an IBM Redbook dealing with specific products or solutions, while getting hands-on experience with leading-edge technologies. You will team with IBM technical professionals, Business Partners and/or customers.

Your efforts will help increase product acceptance and customer satisfaction. As a bonus, you'll develop a network of contacts in IBM development labs, and increase your productivity and marketability.

xviii Large Objects with DB2 for z/OS and OS/390

Find out more about the residency program, browse the residency index, and apply online at:

ibm.com/redbooks/residencies.html

Comments welcomeYour comments are important to us!

We want our Redbooks to be as helpful as possible. Send us your comments about this or other Redbooks in one of the following ways:

� Use the online Contact us review redbook form found at:

ibm.com/redbooks

� Send your comments in an Internet note to:

[email protected]

� Mail your comments to:

IBM Corporation, International Technical Support OrganizationDept. QXXE Building 80-E2650 Harry RoadSan Jose, California 95120-6099

Preface xix

http://www.redbooks.ibm.com/residencies.html

http://www.redbooks.ibm.com/residencies.html


http://www.ibm.com/redbooks/


http://www.redbooks.ibm.com/contacts.html

xx Large Objects with DB2 for z/OS and OS/390

Chapter 1. Introduction

With the incorporation of object-orientation into DB2 for OS/390, you can define new data types and functions. Some of the data objects you want to model may well be very large and complex. The foundation of object-relational extension, introduced with DB2 for OS/390 Version 6 is, on one hand, support for large objects (LOBs); and, on the other, support for user defined functions (UDFs), user defined distinct types (UDTs), and triggers.

For more information on UDFs, UDTs, and triggers, please refer to DB2 for z/OS Application Programming Topics, SG24-6300. This redbook describes the major enhancements that affect application programming when accessing DB2 data on S/390 or z/Series platform, and, while not dealing directly with LOBs, it includes the object-oriented extensions such as triggers, user-defined function and user-defined distinct types, the use of temporary tables, savepoints, and the numerous functions of the SQL language that can help you build powerful, reliable and scalable applications, whether it be in a traditional environment or on an e-business platform.

Large objects can contain pictures, images, text-documents or even movies, and can now be stored directly in DB2 with sizes up to 2 GB per object and 4,000 Terabytes per LOB column. The largest available size in DB2 before LOBs was 32 Kilobytes, for data defined as a varchar.

Normally, large objects are used and manipulated through graphical user interfaces (GUI) via a Workstation. So with the implementation of LOBs, we can exploit the server capacity that DB2 for OS/390 and z/OS provides today while displaying and feeding them through a user friendly interface.

The introduction of these new data types, with their peculiarities, in a traditional environment has some impact on the current, established, administration processes and programming techniques.

In this book we describe the various new data types introduced for LOBs support, and provide useful information on how to design and use them. We offer some examples of their use, programming considerations, and the new processes which are necessary for their implementation, administration, and maintenance.

1

© Copyright IBM Corp. 2002 1

This redbook is structured as follows:

� Large objects with DB2

In Chapter 2 we provide some preliminary information on object orientation, large objects, and the new data types that DB2 V6 has introduced for their support.

� Creating LOBs

In Chapter 3, we provide information on how to define LOBs, how they are structured in terms of DB2 objects, the impact of the CURRENT RULES special register settings, and some techniques in feeding data into the LOBs.

� Using LOBs

In Chapter 4 we discuss topics related to the differences in using LOBs, when compared with using normal DB2 objects — in particular the SQL semantics, LOB locators, locking, and ROWID.

� DB2 Extenders and LOBs

In Chapter 5 we provide a description of how DB2 Extenders utilize LOBs in the z/OS environment. We look at the setup and administration of the AVI and XML Extenders.

� Data Administration with LOBs

In Chapter 6 we document the differences in administering table spaces pertaining to tables containing LOB columns. We look at the DB2 Catalog, we examine the impact on the various utilities, and document the changes introduced for LOBs support.

� Performance with LOBs

In Chapter 7 we discuss topics directly and indirectly related to LOBs performance. We look at LOB materialization, bufferpool and dataspace use, and then we conclude with more performance specific considerations and recommendations.

1.1 Summary of considerationsBefore you start implementing LOBs in your DB2 system, be aware that they are new objects, and that you need planning and preparation work to add the set up needed for a successful LOBs implementation to your current environment. There is some cost associated with introducing LOBs support in a production shop. In this section we summarize our findings with recommendations on how to reduce their impact, we point out their differences from the usual data requirements, and warn about possible misuse.

Stay current with maintenanceLOB support is extremely pervasive and involves all areas of DB2, with repercussions on the operating system, language support, Extenders, data conversion, client/server connectivity, and back-up and recovery. Given the permutations of functions and data definitions, chances are that you may be exploring new territories. Applying current maintenance and prototyping your application will help you in avoiding pitfalls and gaining valuable operational experience.

Client/server solutionSince you are dealing with new large objects, like text or images, it is likely that you need the GUI capabilities of a front end workstation for presenting them. In this type of client/server solution you may also consider the added value of DRDA connectivity for dealing with data conversion, and the file reference variables supported by SQL when feeding data to the host via SQL INSERTs from the distributed DB2. For this type of solution, bandwidth and the speed of your connections become critical, and, with different platforms involved and maybe several geographies, data conversion needs to be verified.

2 Large Objects with DB2 for z/OS and OS/390

Do not misuse LOBsA major recommendation is to use large objects for what they are intended to be: objects with a size bigger than 32 KB. If you use a LOB environment for objects smaller than 32 KB, small LOBs, or SLOBs, you will not benefit from LOBs’ specific design when DB2 accesses them, but you incur some penalties since DB2 assumes that they are real large objects. For instance, you would waste space when storing a column smaller than 32 KB in a LOB column, because a page in a LOB table space can only contain one row worth of data. Also, since LOBs are not physically stored along with other columns of a table, you would create additional overhead when you store frequently accessed columns in a table different from the one containing your other data. Normally, LOB columns are less frequently accessed than other data in your tables.

Logical data designIf you plan to use a LOB column for denormalization of your data, because varchar was not big enough to hold the amount of data, and LOBs are, it may be more effective to reconsider the design. Be aware that using LOBs for denormalization does not correspond to their intended use and may result in ineffectiveness.

MVS considerationsVerify that you have adequate memory, disk space, and processor capacity before introducing the new LOBs application. When accessing LOBs, DB2 can and will activate data spaces. The activation of data spaces will introduce new messages at the system console and it may influence paging, and you need to talk to your MVS system programmer and agree before hand on the set up of the few new, but important, DB2 and MVS parameters. Before you start implementing LOBs, you should evaluate the DSNZPARMs LOBVALA (maximum data space size per user), and LOBVALS (maximum data space size per system), they will help you in gauging the virtual storage utilized by the data spaces created by DB2 when dealing with LOBs.

CURRENT RULESThe value of your CURRENT RULES special register has a major influence when you deal with LOBs. The system default is set to DB2, unless you have specified SQLRULES STD when binding your local plan. You need to evaluate the implication of the value specified for this parameter because it influences the way LOBs are created and dropped. It also has an impact on how applications can deal with LOBs when they use cursors for accessing their values. You may also consider using the CURRENT RULES special register for temporarily overwriting the default value specified for the plan. You need the STD DB2 option active if you want to define objects with your own naming standards.

Application programming challengesProgramming for LOBs is a bit more challenging than accessing data from a standard table. Your application programmers have to deal with the new data types as well as virtual space requirements and possible use of LOB locators. Knowledge of locators, their functional description and their impact on address spaces can be very useful to avoid a large consumption of memory and refrain from creating bottlenecks in your system.

Impact of loggingYou have the option of de-activating logging for LOBs. This makes sense for very large objects which, when inserted or updated, may affect your system with very heavy logging. If you plan to log all changes being made on your LOB values, check back with your database administrator for relieving bottlenecks on DB2’s log data sets by using fast devices and striping. On the other hand, if you do not log, you should be aware of the restrictions imposed

Chapter 1. Introduction 3

on recovery and availability of your LOBs data. Do not define objects as large to simply avoid logging. The disadvantages for your operational data will certainly outweigh the uncertain savings with non-logging. For instance, a full image copy taken with SHRLEVEL CHANGE stays permanently fuzzy without the log.

Backup and recovery scenariosLarge objects are stored in a different way compared with other DB2 objects. When you implement a large object structure in your current environment, backing up all table spaces which are involved in LOB processing (there are at least two per LOB column) to ensure consistency is only one side of the scenario. The other side is recovering a set of table spaces associated to a LOB column for which new table space states have been introduced: the auxiliary warning (AUXW) and auxiliary check pending (ACHKP). These states need to be dealt with by the DBA during well rehearsed recovery scenarios.

LOBs and data sharingIf you plan to introduce LOBs in a data sharing environment, which is becoming more and more popular, have a close look at parameters which can influence your whole data sharing group. Probable limits and values for additional parameters and buffer pools should be set to values that do not interfere with other components of your system environment.

The LOBs are here!If you have no application using DB2 large objects in your environment, and you think you have no LOBs in your DB2 subsystem, you may be surprised to find out that they do already exist in your DB2 catalog if you have DB2 V7 installed. In this release, if you look closely at the catalog tables, you will find LOB tables already defined within DSNDB06. They are filled in by DB2 when you define Java stored procedures. LOBs are also used by DB2 Extenders, optionally by the Image, Audio, and Video Extenders, while the XML Extender defines LOBs as repositories for documents.


Chapter 2. Large objects with DB2

In this chapter we provide some introductory information on object orientation, large objects, and the new data types that DB2 V6 has introduced for their support. This chapter is structured by:

� Object orientation with DB2� Large objects with DB2� The new data types

2


2.1 Object orientation with DB2The object extensions introduced by DB2 UDB for OS/390 Version 6 offer the benefits of object-oriented technology while increasing the strength of your relational database with an enriched set of data types and functions:

� Large objects

The VARCHAR and VARGRAPHIC data types have a storage limit of 32 KB. Although this may be sufficient for small- to medium-size text data, applications often need to store large text documents. They may also need to store a wide variety of additional data types such as audio, video, drawings, mixed text, graphics, and images. DB2 provides new data types to store these large data objects (LOBs) as strings of up to 2 GB in size.

LOBs are well suited to represent large, complex structures in DB2 tables. You can make effective use of multimedia by storing objects such as complex documents, videos, images, and voice.

� User defined types

User defined types (UDTs), like built-in data types, describe the data that is stored in columns of tables where the instances (or objects) of these data types are stored. They ensure that only those functions and operators that are explicitly defined on a distinct type can be applied to its instances.

A distinct type is a user-defined data type that shares its internal representation with a built-in data type, but is considered to be a separate and incompatible type for semantic purposes. For example, you may want to define a picture type or an audio type which have different semantics but use the built-in data type BLOB for their internal representation.

� User defined functions

User defined functions (UDFs), like built-in functions or operators, support manipulation of distinct type instances (and built-in data types) in SQL queries.

The built-in functions that are supplied with DB2 are a useful set of functions, but they may not satisfy all of your requirements. You can write user-defined functions to meet the specific needs for your installation. For example, a built-in function may perform a calculation you need, but the function does not accept the distinct types you want to pass to it. You can then define a function based on a built-in function, called a sourced user-defined function, that accepts your distinct types. You may need to perform another calculation in your SQL statements for which there is no built-in function. In that situation, you can define and write an external user-defined function.

New and extended built-in functions improve the power of the SQL language with about 100 new built-in functions, extensions to existing functions, and sample user-defined functions.

� Triggers

Triggers bring application logic into the database. Triggers automatically execute a set of SQL statements whenever a specified event occurs. These statements can validate and edit database changes, read and modify the database, and invoke functions that perform operations inside and outside the database.

� Extenders

Text Extenders add the power of full-text retrieval to SQL queries by making use of features available in DB2 that let you store unstructured text documents of up to 2 GB in databases.


Text Extender offers DB2 users and application programmers a fast, versatile, and intelligent method of searching through such text documents. Text Extender's strength lies in its ability to search through many thousands of large text documents at high speed, finding not only what you directly ask for, but also word variations and synonyms.

You are not restricted to searching only in text documents stored in DB2 for z/OS databases, you can also search in text documents stored in files.

Text Extender can access any kind of text document, including word-processing documents in their original native form, and offers a rich set of retrieval capabilities including word, phrase, wildcard, and proximity searching using Boolean logic.

At the heart of Text Extender is IBM's high-performance linguistic search technology. It allows your applications to access and retrieve text documents in a variety of ways. Your applications can:

– Search for documents that contain specific text, synonyms of a word or phrase, or sought-for words in proximity, such as in the same sentence or paragraph.

– Do wildcard searches, using front, middle, and end masking, for word and character masking.

– Search for documents of various languages in various document formats.

– Make a fuzzy search for words having a similar spelling as the search term. This is useful for finding words even when they are misspelled.

– Make a free-text search in which the search argument is expressed in natural language.

– Search for words that sound like the search term.

You can use the DB2 Extenders feature of DB2 to store and manipulate image, audio, video, and text objects. The extenders automatically capture and maintain object information and provide a rich body of APIs.

The other extenders in the family let you search for a combination of image, video, and voice data types in one SQL query. They include (AVI) audio, video, and image, as well as XML.

These extenders define new data types and functions using DB2's built-in support for user-defined types and user-defined functions. You can couple any combination of these data types, that is, image, audio, and video, with a text search query.

The extenders exploit DB2’s support for large objects of up to 2 GB, and for triggers that provide integrity checking across database tables ensuring the referential integrity of the multimedia data.

2.2 Large objects with DB2Within today's multimedia application environments the dependency relies on many types of large data objects. Those objects may be large text documents, X-ray images, audio messages, pictures and many other images — you name them they are there.

Table 2-1 shows some representative sizes of various objects. These are outlined just for comparison purposes. Overall they just give you some idea of the storage requirements of various types of objects that you may have to deal with.

Chapter 2. Large objects with DB2 7

Table 2-1 Typical average size for large objects

The data types previously provided by DB2, such as VARCHAR, are not large enough to hold this amount of data, the limit being 32 KB. LOBs support is based on the new set of data types which were introduced with DB2 V6. With large objects support, DB2 stores and manipulates data objects that are much larger.

Starting with V6, DB2 provides three data types to store large data objects (LOBs) which support strings of up to 2 GB in size, well beyond the 32 KB supported by a varchar column. New techniques for storing these huge amounts of data have also been created within DB2. The three new data types allow you to store directly in DB2 tables objects in size of up to 2 GB, and 4,000 Terabytes (TB) per LOB column. Their characteristics depend on the way they are stored in your DB2 subsystem:

� Character Large Objects, called CLOBs

� Binary Large Objects, called BLOBs

� Double Byte Character Large Objects, called DBCLOBs

DB2 also provides new kinds of data types that are used for better storing and manipulating LOBs:

� LOB locators� ROWIDs

2.3 The new data typesIn this section we introduce the new data types available in DB2 for LOB support.

CLOBs A character large object (CLOB) is a varying length string measured in bytes that can be up to 2 GB long. A CLOB is used to store large SBCS or mixed character-based data, such as documents written with a single coded character set identifier (CCSID). A CLOB is considered to be a character string.

Object From To

Bank checks 30 KB 40 KB

Small image 30 KB 50 KB

Large image 200 KB 3 MB

Color image 20 MB 40 MB

Radiology image 40 MB 60 MB

Video 1 GB / hour -

Feature length movie 2 GB -

High resolution video 3 GB / hour -

High resolution movie 5 GB 6 GB

High definition TV 200 MB / second -


Most text documents stored on other platforms cannot be converted to a CLOB value easily because most known editing applications save data using their own format. This format usually contains an interspersed amount of control information used for font types, font sizes and layout purposes. If you want to store this data as a CLOB value and access it via DB2 functions such as SUBSTR or POSSTR, you may have to convert the data from their format on your PC to a plain text file before inserting into a CLOB column. You may consider using a CLOB column for storing large text documents or even Extended Markup Language (XML) documents. If you plan to store your PC documents in DB2 for z/OS without converting them, because you want to access them from your client applications for further processing, you need to store them as BLOBs to avoid loss of font and layout control information.

BLOBsA binary large object (BLOB) is a varying length string measured in bytes that can be up to 2 GB long. A BLOB is primarily intended to hold non-traditional data such as pictures or images, sounds or voices, or other types of mixed-media. All these objects eventually consist of unreadable binary data strings. When you create BLOBs, the data is stored in a LOB column as it is passed to DB2, without any data conversion. The same rule applies when you retrieve the data. Differently from CLOBs, the values stored in BLOB columns do not go through conversion routines when accessed by DB2.

Another use of BLOBs is to hold data for exploitation by distinct types and user-defined functions.

Normal character strings defined with the FOR BIT DATA option cannot be assigned a CCSID, and the same rule applies to BLOB strings.

A BLOB contains a binary string representing binary data, a sequence of bytes, normally being unreadable. Common character strings defined FOR BIT DATA, which may be used for similar purposes, are not compatible with data type BLOB. However, the CAST specification can be used to cast a character string to a binary string.

DBCLOBsA double-byte character large object (DBCLOB) is a varying length string of double-byte characters that can be up to 2 GB long. Using a DBCLOB column you can store up to 1,073,741,824 double-byte characters in a single DBCLOB value. A DBCLOB is used to store large double byte character set (DBCS) character-based data such as documents written with a double-byte CCSID. A DBCLOB is considered to be a graphic string. You can find double-byte CCSIDs for languages like Japanese (extended Katakana or Katakana-Kanji), Korean, or Chinese (simplified or traditional).

2.3.1 LOB locatorsIn general, when the application is accessing data, DB2 has to deal with the materialization of data across several layers: auxiliary storage, central storage, data spaces, and the movement of data across them. The materialization of LOB values consists of placing LOB data in contiguous storage. Since LOB values can be very large, DB2’s actions are inspired to the objective of avoiding materialization of LOB data until it becomes absolutely necessary. Furthermore, if the application is running on a client machine, distinct from the database server, the transfer of LOB values from the server to the client adds a sizeable time and resource consumption. Application programs typically process LOB values a piece at a time, rather than as a whole. For all those cases where an application does not need (or want) the entire LOB value to be stored in application memory, the application can reference a LOB value via the new large object locator (LOB locator) and avoid materialization.


A LOB locator is a host variable with a value that represents a single LOB instance in the database server. LOB locators have been developed to provide users with a mechanism by which they could easily manipulate very large objects in application programs without requiring them to store the entire LOB value in the application’s memory.

You can consider locators roughly as cursors that reference remote data. Locators, and the structures associated to them, are contained in virtual storage blocks allocated by DB2 in the DBM1 address space. The LOB data referenced by the locators is allocated by DB2 in data spaces which are dynamically created for the duration the request.

Purpose of LOB locatorsA LOB locator is a value generated by DB2 when a LOB string is assigned to a host variable that was previously specified as a LOB locator in your application program. Every LOB locator, also called host-identifier, is a special type of SQL object materialized as a four byte token used to represent the LOB. This representation allows the value of the large object strings to be contained in the host-identifier, rather than the actual string of bytes of the large object. Figure 2-1 provides an example of LOB locator usage.

Figure 2-1 Assigning a LOB locator for a LOB value

A LOB locator is an association between a host variable and its corresponding LOB value in DB2 at a point in time. The application only accesses the locator while the entire LOB value resides in DB2 and is not propagated to the application program. Using this technique, an application does not need to acquire a buffer large enough to contain the entire LOB value. Instead the application deals only with LOB locators therefore largely reducing the amount of resources to be allocated. The definition and usage are not mandatory; however, considerations on performance will soon lead you to the conclusion that it is a good idea for the applications to use them, since locators dramatically reduce the movement of data between the different address spaces involved in LOB data management as well as greatly reducing connectivity issues.

1 GB BOOK_TEXT

SELECT BOOK_TEXT INTO :BOOK_TEXT_LOCATOR FROM BOOK_BASE_TABLEWHERE BOOK_NO = :HV_BOOK_NO

:BOOK_TEXT_LOCATOR

DB2Application

Internal pointer

987654321987654321


Different types of LOB locatorsEach LOB format (BLOB, CLOB and DBCLOB) has its own type of locator, so currently there are three different types of LOB locators:

� Binary Large Object locator, which is associated with large binary strings� Character Large Object locator, which is associated with large character strings� Double Byte Character Large Object locator, which is associated with large graphic strings

The definition of a LOB locator depends on the language you choose for developing programs for LOBs usage. The syntax for defining LOB locators in COBOL is shown in Example 2-1.

Example 2-1 Host variable definitions for LOB locators in COBOL

01 BLOB-LOCATOR USAGE IS SQL TYPE IS BLOB-LOCATOR.01 CLOB-LOCATOR USAGE IS SQL TYPE IS CLOB-LOCATOR.01 DBCLOB-LOCATOR USAGE IS SQL TYPE IS DBCLOB-LOCATOR.

The DB2 precompiler converts the locator structure into the COBOL structure as reported in Example 2-2.

Example 2-2 What the DB2 precompiler makes of LOB locators

01 BLOB-LOCATOR PIC S9(9) COMP.01 CLOB-LOCATOR PIC S9(9) COMP.01 DBCLOB-LOCATOR PIC S9(9) COMP.

The locator 4-byte value is then stored in a host variable; the program, as already shown in Figure 2-1, can use it to refer to a LOB value. Even if every LOB locator shows up identical for all definitions of host variables, DB2 knows the associated LOB type and will not let you use a locator with a different type of LOB. If you define a CLOB locator and try to use it for a BLOB, SQLCODE -171 is issued.

You can only use LOB locators inside an application program, you cannot deal with them interactively. This means that you cannot use them, for instance, with SPUFI or QMF.

For more information on LOB locators refer to 4.2, “LOB locators” on page 46.

2.3.2 The ROWID The ROWID data type (and column) definition was introduced with DB2 V6 to uniquely and permanently identify a row in a table. To understand the role of the ROWID, and why DB2 creates a value for a ROWID column whenever a row is inserted in a table containing this new data type, you must first look at and understand the overall picture of the DB2 objects involved in supporting LOBs, as illustrated in Figure 2-2.

LOB data is contained in a LOB column which is conceptually part of the base table, but it is physically stored in a separate table. Because it is not part of the base table, it is called LOB table or auxiliary table. The auxiliary table resides in a separate LOB table space.

A base table may be associated with many LOB or auxiliary columns of different types and lengths. Each auxiliary column is stored in its own auxiliary LOB table in its own LOB table space. An auxiliary index must be created on every auxiliary table before it can be used. To create a base table that contains a LOB column, you must define a ROWID column. The ROWID acts as, but it is not, a bidirectional pointer to the LOB data associated with the particular row: the LOB column values are associated with the proper base table row in both directions using the base table row’s ROWID value.The auxiliary index, whose key is based on the ROWID, is used to navigate to LOB data associated with the row.


If a base table that contains LOB data is partitioned, you create a separate LOB table space and auxiliary table for each LOB column in each partition. A LOB table space can have a page size of 4, 8, 16 or 32 KB. Since the length of a LOB can exceed 32 KB, it is clear that a LOB can span physical pages, and since there is only one row per page, with LOBs, rows can span pages. LOB pages only contain one LOB value, or row.

Figure 2-2 LOB structure

More information about ROWIDs is reported in 4.6, “Details about ROWID” on page 68.

Base table space

Key ROWID Column_2 LOB indicator

Key A LOB 1 value user data A LOB indicator 1Key B LOB 2 value user data B LOB indicator 2

Base table

Auxiliary index:based on ROWID used to navigate to LOB data

LOB table space

ROWID LOB dataLOB 1 value LOB data for row user data ALOB 2 value LOB data for row user data B

Auxiliary table

Rows represent LOBs LOBs stored outside base table in auxiliary tableBase table space may be partitionedIf so, separate LOB table space for each part


Chapter 3. Creating LOBs

In this chapter, we provide information on how to define and load LOBs. We discuss the following topics:

� Data conversion � Storing LOBs � CURRENT RULES special register� Feeding a LOB column

3


3.1 Data conversionDB2 for z/OS is often used as the enterprise server of large client server systems. In these environments, character representations may vary on clients and servers across different platforms and across many different geographic locations. In this scenario you may also want to deal with data conversion topics. Large objects can also be subject for conversion topics, depending on the type of data you plan to store in your large objects. See Figure 3-1.

Figure 3-1 Mixed client /server environment with different data types

One area where this sort of environment exists is in data centers of multinational companies. Another example is e-commerce. In both of these examples, a geographically diverse group of users interact with a central server, storing and retrieving data.

Today, there are hundreds of different encoding systems. No single encoding could contain enough characters: for example, the European Union alone requires several different encoding schemes to cover all its languages. Even for a single language like English, no single encoding was adequate for all the letters, punctuation, and technical symbols in common use.

These encoding systems also conflict with one another. That is, two encoding schemes can use the same number for two different characters, or use different numbers for the same character.

Mixed client/server environment

CCSID273

CCSID850

CCSID500


DB2 V5 introduced support for storing data in ASCII. This support only solved part of the problem: padding and collating. It did not address the problem of users in many different geographies storing and accessing data in the same central DB2 server. With DB2 V6 and LOBs, chances are that LOBs are being moved to the host from other platforms or other geographies, and a verification of the needed data conversion is recommended.

As we mentioned in 2.3, “The new data types” on page 8, a LOB is a string data element just like the more familiar string data elements, such as CHAR or VARCHAR, except that a large object may be as much as 2 GB long. The three new data types that are used to identify large objects are CLOBs, BLOBs, and DBCLOBs.

BLOBs are designed to contain binary data. As such, they have no CCSID associated with them. CLOBs and DBCLOBs are designed to contain text data. CLOBs have the normal single byte and mixed CCSIDs associated with them, while DBCLOBs have the graphic CCSID associated with them.

For encoding and decoding, DB2 uses the table SYSIBM.SYSSTRINGS for conversion to and from CCSIDs. This has not changed from the past releases of DB2.

Within DB2 V7 there is Unicode support. This also requires OS/390 Version 2.8 or above. Example 3-2 gives an idea of the impact using Unicode.

Figure 3-2 Unicode considerations

Unicode provides a unique number for almost every character, on any platform, in any language, and by any program. The Unicode character encoding standard is a character-encoding scheme that includes characters from almost all living languages. It is an implementation of the ISO-10646 standard.

United States

France

Germany

DB2 Unicode supportJapan

China

Chapter 3. Creating LOBs 15

There are several popular implementations of the Unicode standard such as:

� UCS-2 - Universal Character Set coded in 2 octets.

� UCS-4 - Universal Character Set coded in 4 octets. This will become UTF-32.

� UTF-8 - UNICODE Transformation Format for 8 bit (ASCII safe UNICODE). Characters are encoded in 1 to 6 bytes.

� UTF-16 - UNICODE Transformation Format for 16 bits. The format is a superset of UCS-2 and contains an encoding form that allows more than 64 KB characters to be represented.

DB2 V7 support for UNICODE provides the most popular implementations of UNICODE: UTF-8 and UTF-16.

� CHAR, VARCHAR, LONG VARCHAR and CLOB data for SBCS data is stored as ASCII (7 bit) CODE CCSID 367

� CHAR, VARCHAR, LONG VARCHAR and CLOB data for mixed data is stored as UTF-8 (UNICODE CCSID 1208)

� GRAPHIC, VARGRAPHIC, LONG VARGRAPHIC and DBCLOB data is stored as UTF-16 (UNICODE CCSID 1200)

If you are working with character string data in UTF-8, you should be aware that ASCII characters are encoded into one byte lengths. However, non-ASCII characters, for example Japanese characters, are encoded into two or three byte lengths in a multiple-byte character code set (MBCS). Therefore, if you define an 'n' bytes length character column, you can store strings anywhere from 'n/3' to 'n' characters depending on the ratio of ASCII to non-ASCII character code elements. DB2 does not use the table SYSIBM.SYSSTRINGS for conversion to and from UNICODE CCSIDs. Instead DB2 uses OS/390 Conversion Services, a feature shipped with OS/390 Version 2 Release 8, to manage all the conversions to and from UNICODE CCSIDs.

Setting up the new UNICODE support using OS/390 Conversion Services needs some planning and investigation before you enable it. Appendix B, “Unicode implementation” on page 147 provides an example of how to set it up.

3.2 Storing LOBsThis section gives you an idea of how LOBs are stored in DB2 V7. We provide you with sample DDL needed for the creation of all LOB dependent objects, including a base table, a LOB table space, an auxiliary table, and an auxiliary index. We also discuss the way DB2 stores the data at page level and how buffer pools and data spaces are associated with a LOB table space.

3.2.1 How to store them in DB2CLOBs, BLOBs, and DBCLOBs are the new data types provided by DB2 for large objects support. DB2 also provides new techniques to accommodate the storing of these possibly huge amounts of data within the DB2 structures. In fact a large object in DB2 can be compared to a more familiar string element, such as a CHAR or a VARCHAR column with the main difference being that these LOB columns can reach the size of 2 GB minus one byte (2,147,483,647 bytes). This requires different handling techniques.


Generally, non-LOB columns are stored in what we refer to as a base table. LOBs belong to a base table and are related to it, but they are not stored in the same table with the other non-LOB columns, they are stored in a new type of table space, which is called a LOB table space. See Figure 3-3.

Figure 3-3 Association between base table and auxiliary table

Each LOB column in a base table requires its own LOB table space, so LOB columns are also separated from other LOB columns belonging to the same base table. It is important to understand that a page never contains values of two LOB columns. For a pictorial view of the different LOB columns and its associated LOB table spaces see Figure 3-4.

Figure 3-4 Base table with two LOB columns and the two associated LOB table spaces

Col 1 Col 2 LOB Col 1

Vala Valb

Valc Vald

Base Table Space Base Table

Picture1

Picture2

LOB Table SpaceAuxiliary

Table

Col 1 Col 2 LOB Col 1 LOB Col 2

Vala Valb

Valc Vald

Base Table Space Movie1

Movie2

LOB Table Space 2Auxiliary

Table

Picture1

Picture2

LOB Table Space 1

Base Table

AuxiliaryTable


If the base table is partitioned, every LOB column in every partition has its own LOB table space. This means up to 254 LOB table spaces per LOB column if the base table has 254 partitions. Figure 3-5 gives you a better understanding of a multi-LOB column base table and the required LOB table spaces.

Figure 3-5 Partitioned base table containing two LOB columns

The table where the rows of a LOB column live, and which is embedded in the LOB table space, is called an auxiliary table (aux table). The LOB column itself is referred to as an auxiliary column, because it is not stored in the base table. The rows in the base table are associated to the LOB columns residing in the auxiliary table in the LOB table space using the ROWID. For further information about ROWIDs, see 4.6, “Details about ROWID” on page 68.

In order to quickly locate the proper LOB value in the auxiliary table, an auxiliary index must be created on the auxiliary table. This index includes the ROWID. Additionally a LOB indicator is also stored in the base table. You can find more information about this indicator in “A few more details on the base table” on page 20. See Figure 3-6 for a summary picture of all the objects mentioned here and how they work together.

Movie1

Movie2

LOB Table Space 2Auxiliary

Table


Vale Valf

Valg Valh


Vala Valb

Valc Vald

Partition 1 - Base Table Space

Picture1

Picture2

LOB Table Space1

Movie3

LOB Table Space 4

LOB Table Space 3

AuxiliaryTable

Picture4

Picture3

AuxiliaryTable

AuxiliaryTable

Partition 2 - Base Table Space

Base Table

Base Table

Movie4


Figure 3-6 Association between base table, ROWID, auxiliary table, and LOB table space

An important reason for the separation of LOB columns is performance. The decision was made during the design phase of large objects support, based on experiences using middleware systems and LOBs. Assuming table space scans on the base table, LOB values do not have to be processed during these scans. Probably most of the scanning time would be spent in scanning LOB columns if they would reside in the same table as the non-LOB columns. DB2’s Data Manager (DM) only handles column-sizes up to 32 KB. For this purpose a new LOB Manager (LOBM) was introduced in DB2 V6.

Based on the nature of LOBs, they can be larger than the biggest available page size (32 KB in DB2 V6 and V7) which is also valid for LOB table spaces. Therefore a single LOB value can span pages. For further information on storing LOBs, see 4.4.1, “Storage of LOBs” on page 60.

Because each data set of a LOB table space may grow up to 64 GB, and there may be up to 254 data sets per table space, the maximum size of a non-partitioned LOB table space is 16,256 GB (16 TB) in total. As the number of partitions can grow up to 254 in the base table, there will be 254 single LOB table spaces, each holding up to 16 TB of data as a maximum size. This gives a grand total of 254 x 16 TB = 4,064 TB available for a single column of LOB data.

3.2.2 Creating a table containing LOBsIn our example we assume that we need to set up a new table containing two non-LOB columns, a CLOB, and a BLOB column. The base table is stored in the base table space.

Col 1 Col 2 LOB Col 1 LOB Col 1 Flags

Vala Valb ABCDEFG... ...

Valc Vald 1234567... ...

Base Table Space

Base Table

Row ID

Row ID LOB Value

ABCDEFG... Picture 1

1234567... Picture 2

LOB Table Space

AuxiliaryIndex

Auxiliary Table


The first step is creating the base table in a table space already existing in the same database where the LOB table spaces will be stored. There are no special suggestions for a base table space: it is just a normal table space. But we recommend that you have only one base table in a base table space. This will simplify your recovery procedures by dealing independently with each LOB table space.

As an example, we create a base table containing information about a book, storing the book itself as a CLOB, and the image of the book cover as a BLOB.

The base table is created with the DDL statement reported in Example 3-1.

Example 3-1 DDL for a base table

CREATE TABLE BOOK_BASE_TABLE( BOOK_NO CHAR(10) NOT NULL WITH DEFAULT, DESCRIPTION CHAR(32) NOT NULL WITH DEFAULT, ROW_ID ROWID NOT NULL GENERATED ALWAYS, BOOK_TEXT CLOB(500M) NOT NULL, BOOK_COVER BLOB(1M) NOT NULL );IN LOBDB.BASETS;

The only supported default value for a LOB column is NULL.

For CLOBs you can also specify the parameters FOR SCBS, MIXED or BIT DATA. A CCSID EBCDIC or ASCII may also be specified for CLOBs. For BLOBs and DBCLOBs this is not supported, because BLOBs contain binary data and DBCLOBs have a graphic CCSID associated with them.

Sizes of LOBs can be assigned in kilobytes (K), megabytes (M) or even gigabytes (G). For syntax on CREATE TABLE statements regarding LOBs, refer to DB2 UDB for OS/390 and z/OS Version 7 SQL Reference, SC26-9944.

A few more details on the base tableThe additional column containing the new data type ROWID simply contains unique values related to the auxiliary tables in the LOB table spaces. A column of data type ROWID has to be defined within your base table when you use LOBs in your base table.

The data type ROWID is stored as a VARCHAR (17) column in the base table. It is implicitly one part of the unique index for each auxiliary table containing the LOB columns in the LOB table spaces, where the LOB columns are stored.

Even if a base table contains more than one LOB column, only one ROWID column is needed. So, all LOB columns in one row are associated with the same ROWID value. Regarding this procedure, the ROWID column is a unique and permanent identifier for each row in the base table. A ROWID is a new data type and is not the same as a record identifier (RID), which is internally used by DB2 to reflect the position of a row in a table. But you can find the RID as a part of the externalized ROWID when you select the ROWID column.

There are two ways of assigning a ROWID to a base table.

� The first one is specifying GENERATED ALWAYS in the DDL. Using this syntax, DB2 takes care of the ROWID column and users do not have to take any action regarding ROWIDs. So the ROWID column will not appear in the VALUES clause of an INSERT

Note: The table space containing the base table has to be in the same database as every associated LOB table space.


statement. The recommendation is to use GENERATED ALWAYS unless you want to copy data from other tables into your table.

� The second option consists on specifying GENERATED BY DEFAULT. Using this syntax, DB2 generates a value only if you do not specify any value in the VALUES clause of an INSERT statement. This is intended to be used only for copying data from other tables. Applications or users should never try to generate ROWID values, because DB2 performs validity checking while inserting a row in the specified table. When you plan to use GENERATED BY DEFAULT be aware that DB2 requires a unique index on the ROWID column on the base table.

More information on ROWIDs is reported in 4.6, “Details about ROWID” on page 68.

In the table definitions of the new data types CLOB (500 MB) and BLOB (1 MB) in Example 3-1 on page 20, DB2 will implicitly put two LOB indicators into the base table definition. Only indicator columns are stored in the base table in place of the actual LOB columns.

A LOB indicator for a LOB column consists of six bytes, and it provides useful information about the stored LOB to DB2 when it accesses the column. The LOB indicators are stored like VARCHAR (4) columns. They are made up of 2-bytes flag and 2-bytes string containing the number of the currently stored version of the LOB. The flag bytes contain a NULL flag which has the information about a NULL, or even a NOT NULL value assigned to a specific LOB value. Another information retrieved from the flag bytes is the zero length flag which indicates that a LOB column does not contain any data. Invalid LOB values are also marked invalid using the flag bytes. By referring to this information, DB2 does not have to read the auxiliary table when any of these conditions is true. Figure 3-7 illustrates the catalog information for the created table.

Figure 3-7 Catalog description for table BOOK_BASE_TABLE

The remaining two bytes contain the current version of the LOB value. This is stored to detect mismatch situations between the base table and the auxiliary table. For further information on possible mismatches, see 6.2.8, “CHECK LOB” on page 104.

Trying to access the base table now, by using SELECT, UPDATE, INSERT or DELETE, results in DB2 issuing SQLCODE -747. Access to the base table is not allowed, because the base table is marked as incomplete if at least one dependent object is not defined.

Catalog description

Stored asVARCHAR (17)

Book_No Description ROW_ID Book_Text Book_Cover

CHAR (10) CHAR (32) ROWID (17) CLOB (4) BLOB (4)




The information when a table definition is incomplete is stored in column TABLESTATUS in SYSIBM.SYSTABLES introduced in DB2 V6. A value of ‘L’ indicates a missing auxiliary index or auxiliary table for a LOB column. When the content of the value is ‘R’, you have used the GENERATED BY DEFAULT clause for your ROWID column and have not yet created the required single column unique index on that particular column in your base table.

If the CURRENT RULES special register is set to STD, DB2 creates the dependent objects by itself during the processing of the base table DDL. The naming of the automatically created objects are determined by DB2’s default algorithms. We talk about CURRENT RULES in 3.3, “The CURRENT RULES special register” on page 31. You can use the REPORT utility to find out which names DB2 has assigned to your auxiliary objects, but you can also have a look at the catalog as usual. For more information about the REPORT utility, see 6.2.5, “REPORT TABLESPACESET” on page 99.

3.2.3 Creating the LOB table spacesAfter the base table is created, we define the LOB table space containing the LOB column. In our example we need two LOB table spaces, one for the CLOB column, the other one for the BLOB column.

Both LOB table spaces can be created using the sample statement shown in Example 3-2.

Example 3-2 DDL for a LOB table space

CREATE LOB TABLESPACE BLOBATS1IN LOBDBUSING STOGROUP BLOBSGTS PRIQTY 1000000 SECQTY 1000000LOG NOLOCKSIZE LOBBUFFERPOOL BP32KDSSIZE 64GGBPCACHE SYSTEM;

The keyword LOB tells DB2 to create the table space with the new format. You cannot store LOB values in any other than LOB table space (for instance the generic LARGE table space). Specifying free space parameter (FREESPACE and PCTFREE) has no influence with LOBs.

The second LOB table space holding the CLOB column is created in a similar way.

Every LOB column needs its own LOB table space. Partitioning of LOB table spaces is not allowed, but they are divided in pagesets in accordance with the partitioned base table definition.

Compression is not supported for LOB table spaces. For a more detailed description of how the data is stored in a LOB table space, see 4.4.1, “Storage of LOBs” on page 60.

Note: You cannot access the base table via SQL until all dependent objects (LOB table space, auxiliary table, and the auxiliary index) are defined.


Buffer pools and LOB table spacesIf you want to store huge amounts of data in your LOBs, you may consider using separate storage groups and buffer pools from your other data in order to avoid interference resulting in a possible impact on performance. This is because a single LOB value is able to use up to 2 GB of your buffer pool if assigned to the same buffer pool as the non-LOB table spaces. If all LOBs are going to be read, the content of the buffer pool may mainly consist of LOBs.

Using LOG YES or LOG NOLogging LOB data can create a bottleneck in your DB2 subsystem, because each logged LOB can grow up to 1 GB in size and the amount of data to be logged grows according to the actual size of the manipulated LOBs. DB2 does not allow you to turn on logging for LOBs bigger than 1 GB.

Specifying LOG YES for a LOB table space tells DB2 to log almost all data manipulations on LOB columns stored in the associated LOB table space, except delete operations. When you delete a LOB value, no LOB data is written to the log, only LOB system pages are written to the log data sets, because a LOB delete internally is translated into only de-allocation of the pages where the LOB is stored. The de-allocation of a LOB is flagged in the space map pages for all pages containing the LOB information, including the LOB map pages. For more information about the structure of a LOB table space, refer to 4.4.1, “Storage of LOBs” on page 60.

When you insert a LOB, DB2 writes the entire LOB value to the log. The entire LOB is also written to the log even when you update a LOB value, because updates consist of one singleton delete (where DB2 does not write data in the log) and one insert into the auxiliary table.

Depending on the size of your LOBs and the frequency you regularly insert or update them, the data to be logged may grow rapidly. When you plan to use LOG YES, make sure that writing the redo logs will not become a critical factor!

To prevent your system from the probable bottleneck caused by logging large LOB data, DB2 allows you to turn off logging for large objects. LOG NO simply tells your DB2 subsystem to suppress writing redo logs for every LOB column in your LOB table space. The force at commit protocol ensures that LOB values persist after a unit of work is committed, because LOG NO LOB values are written at COMMIT. LOB data associated with a LOB table space defined with LOG YES is written to disk as usual for other data when buffer pool thresholds are reached.

If you specify LOG YES for a LOB table space where the LOB size exceeds 1 GB, DB2 issues SQLCODE -355 when creating the auxiliary table. This indicates that a LOB column is too large to be logged. Keep in mind that LOG YES is the default value when you create your LOB table space without specifying the LOG clause.

You can find a detailed description of LOG impact between LOG YES and LOG NO in 7.3.1, “Logging with LOBs” on page 128. For information about the logging impact when running the Load utility, see 6.2.1, “LOAD” on page 94.

Note: LOBs larger than 1 GB cannot be logged. Changes to the base table are not affected by specifying the LOG clause on the LOB table space.


DB2 rollback without undo log recordsAs new pages are allocated while inserting a LOB, they are simply de-allocated if DB2 does a rollback. If a LOB is deleted, pages are also simply re-allocated as available in the LOB table space. Since updating a LOB means deleting a LOB (de-allocating pages) and inserting a new one (allocating new pages), at rollback time already allocated pages are de-allocated and previously de-allocated pages are re-allocated again. This mechanism is known as Shadow Copy Recovery, see “Shadow Copy Recovery” on page 67.

Because of allocation and de-allocation of data pages, no UNDO logs are written for LOBs, regardless what LOG parameter you use.

Header pages, space map pages, and the new LOB map pages (see “LOB table space organization” on page 61) are logged even if LOG NO is specified. They are backed out using the same procedure used for other types of table spaces. The same rule applies if an application terminates abnormally after changing the database. Even in this case, those changes made in the current unit of work are backed out along with all other changes, too.

If LOG NO is specified, DB2 does not log any changes applied to any LOBs in the associated LOB table space. Therefore recovery of LOB table spaces can be more difficult than recovering any other table space. Let us assume the scenario depicted in Figure 3-8.

Figure 3-8 Applying changes to a LOB table space created with LOG NO

A full Image Copy is taken of a base table space and the associated LOB table spaces. After completion of the full Image Copies, one LOB is inserted, another one is deleted and one more is updated. Assume now that you have to recover the base table space and the LOB table space after all updates are done because of a system failure. For the base table DB2 can apply all necessary changes from the log. Recovering the LOB table space for DB2 is more difficult with a specified LOG NO option. After using the full image copy, DB2 notices that a new LOB was inserted without writing any logs about its value. So the new LOB is marked invalid in the auxiliary table. The next important log point is deleting LOB number two. DB2 knows from the logged system pages what has happened and marks the pages formerly used by LOB number two as available. The next point on the log brings us to a LOB update. Since UPDATE consists of DELETE and INSERT, the pages are deallocated in the LOB table space and for not having a log to apply for the new LOB value, it is also marked as invalid in the LOB table space. In our case the LOB table space is set in a new AUX WARNING (AUXW) state. Trying to access an invalid LOB value results in SQLCODE -904. For more information on recovery scenarios, see 6.2.13, “RECOVERY” on page 109.

Col 1 Col 2 LOB Col 1

Vala Valb

Valc Vald

Base Table Space Base Table

Picture1

Picture2

LOB Table SpaceAuxiliary

Table


With DB2 V6 and V7, the LOG NO clause can be provided for LOB table spaces, but no other types of table spaces support this option.

Locking for LOBsThere are at least two different types of lock sizes you can choose between. Valid parameters are LOCKSIZE LOB and LOCKSIZE TABLESPACE. Choosing LOCKSIZE ANY implies the use of LOCKSIZE LOB for DB2. Two new types of locks were introduced for LOBs in DB2 Version 6: the S-LOB and the X-LOB locks. They are very similar to the common S- and X-Locks. There are no U-LOB locks because of a different update mechanism taking place for LOBs. See “Using LOG YES or LOG NO” on page 23 for a description of updating techniques for LOBs.

There is no support for Uncommitted Read (UR) on LOBs. Assuming LOCKSIZE LOB, selecting a LOB acquires an S-LOB lock on the accessed LOB, even if you use ISOLATION (UR) in a package or WITH UR in your SQL statement. This is because of the new table space format where physical correctness has to be guaranteed while retrieving the LOB value, which may be spanned over many pages. Acquiring S-LOB locks prevents the application from retrieving only partial LOB data.

Deleting a LOB also requests an S-LOB lock on a specific LOB. But how does this work with other transactions selecting the same LOB? A transaction can delete a LOB that another transaction is reading at the same time, but the space is not reused until all readers of the LOB have committed their work.

So S-LOB locks do not prevent a LOB from being deleted.

Inserting a LOB acquires an X-LOB lock on the new LOB, the lock is released at COMMIT as usual. If a single LOB is locked by an X-LOB lock, no readers can access them before it is committed.

As every LOB lock is like a row lock, the number of acquired locks can increase dramatically, especially when mass-updates occur on LOBs, or many LOBs are inserted via subselect.

You can find more details about locking for LOBs in 4.5, “Locking” on page 62.

Buffer pool and page size considerationsYou can assign 4 KB, 8 KB, 16 KB or 32 KB buffer pools to your LOB table space. But which one to use depends on several considerations.

Choosing a page size is the common trade-off between minimizing the number of getpages, which simply maximizes performance, and not wasting space. Because one data page in a LOB table space never stores data of more than one LOB value, the space not used by the LOB value in the last page remains unused. To estimate a good average size of a LOB value, use the formula:

Size of a LOB = (Average length of all LOBs) * 1.1

Use the rule-of-thumb shown in Table 3-1 to choose a page size that minimizes the number of getpages.

Attention: Choose Image Copies with SHRLEVEL REFERENCE when using LOG NO, and define carefully when to take them based on your application needs. Image Copies using SHRLEVEL CHANGE allow updates to occur, and with a LOG NO table space, you have unusable copies.


Table 3-1 Choosing a LOB page size that minimizes getpages

Using this technique, a LOB value of 22 KB means 10 KB of unused space, but only one getpage request for DB2. Therefore you have to analyze your data to determine what is best for you and your LOBs. Normally, you have 32 KB data pages assigned to your LOB table space, because you should use LOBs only for real large objects.

If your LOB data is all the same size, it may be easier to choose a page size that makes more efficient use of data pages without impacting performance. For LOBs that have all the same size, consider Table 3-2, which maximizes space savings without sacrificing performance.

Table 3-2 Choosing a LOB page size for LOBs all of the same size

Also see buffer pool considerations and buffer pool thresholds (DWQT and VDWQT) in 7.2, “Buffer pools and group buffer pools” on page 125 for more detailed information on this topic.

If you have decided the page size you want to use, DB2 allocates at least n KB of space even if you tell DB2 to allocate less than the minimum amount of space for that page size. Table 3-3 provides information of the minimum space requirements for primary (PRIQTY) and secondary (SECQTY) quantity and how they may change from your specification.

Table 3-3 Primary and secondary quantity with LOBs

Average LOB size (ALS) Recommended page size for LOB table space

ALS ≤ 4 KB 4 KB

4 KB < ALS ≤ 8 KB 8 KB

8 KB < ALS ≤ 16 KB 16 KB

16 KB < ALS 32 KB

LOB size (LS) Recommended page size for LOB table space

LS ≤ 4 KB 4 KB

4 KB < LS ≤ 8 KB 8 KB

8 KB < LS ≤ 12 KB 4 KB

12 KB < LS ≤ 16 KB 16 KB

16 KB < LS ≤ 24 KB 8 KB

24 KB < LS ≤ 32 KB 32 KB

32 KB < LS ≤ 48 KB 16 KB

48 KB < LS 32 KB

PRIQTY and SECQTY

Specification in KB Page size Resulting allocation in KB

< 200 4 KB 200

< 400 8 KB 400

< 800 16 KB 800

< 1,600 32 KB 1600

> 4,194,304 Any 4,194,304


Need to specify DSSIZE for LOB table spacesDSSIZE represents the data set size value passed to DB2 to indicate the maximum allowed size. Specifying DSSIZE 64 GB allows one data set of size up to 64 GB.

Table 3-4 summarizes the partition and partitioned table space sizes for the current DB2 versions.

Table 3-4 Summary of data set, partition, and partitioned table space sizes

Up to DB2 V4 the total maximum size of a partitioned table space was 64 GB. Starting with DB2 V5, with the introduction of the LARGE parameter at creation time, partitioned table spaces may have a total size of 1,016 GB, corresponding to up to 254 partitions each with a data set size of 4 GB. DB2 V6 introduced the DSSIZE parameter to replace LARGE, and in combination with the extended addressability function of SMS managed VSAM data sets, can extend the data set size to 64 GB.

Since each LOB table space may consist of 254 data sets, you can reach the maximum amount of 16,256 GB (approximately 16 TB) for a single LOB column in one LOB table space. Considering a partitioned base table with 254 partitions, these 16,256 GB can occur up to 254 times (one LOB table space for each partition of the base table), so you can have 4,129,024 GB or 4,129 TB for one single LOB column.

The default value passed to DB2, if DSSIZE is not specified, is 4 GB per data set. This means a maximum value of 1,016 GB (approximately 1 TB) for a non-partitioned base table and 258,064 GB (approximately 258 TB) for a partitioned base table consisting of 254 partitions.

The maximum amount of space for PRIQTY and SECQTY you can specify is at least 4,194,304 KB. So make sure that your specified PRIQTY and n times SECQTY, if allocated, is about 64 GB. The value of n depends on the version of DFSMS running in your system. Starting with DFSMS Version 1 Release 4, 254 extends are supported. Extended addressability for VSAM data sets, leading to a maximum file size of 64 GB, was introduced in DFSMS Version 1 Release 5. To benefit from the maximum size provided for LOBs, DFSMS Version 1 Release 5 is required.

GBPCACHE parameterA new GBPCACHE SYSTEM parameter was added for LOBs to prevent LOB data from flooding your group buffer pool. Specifying GBPCACHE SYSTEM caches only the LOB control (system) information in the group buffer pool.

In a data sharing environment GBPCACHE SYSTEM is recommended for large objects.

See 7.2, “Buffer pools and group buffer pools” on page 125 for more information.

DB2 Version Number of partitions Maximum size each Total maximum size

V5* 254 4 GB 1,016 GB

V6 and V7** 254 64 GB 16,256 GB

Note: * Requires LARGE parameter in CREATE TABLESPACE

Note: ** Requires DSSIZE and DFSMS/MVS 1.5 and SMS-managed table space

Note: If DSSIZE is greater than 4 GB, make sure that this data set belongs to a DFSMS class that is defined with the extended addressability attribute, and also that the automatic class selection routine associates this data set with this data class. Otherwise DB2 is not able to allocate the requested space, and it will issue SQLCODE -904.


3.2.4 Creating the auxiliary tablesThe third step is creating an auxiliary table, holding the data of a LOB column. There can only be one auxiliary table per LOB table space.

The DDL for creating an auxiliary table in a LOB table space is shown in Example 3-3.

Example 3-3 DDL for an auxiliary table

CREATE AUXILIARY TABLE BLOB_AUX_TABLE_1 IN LOBDB.BLOBATS1 STORES BOOK_BASE_TABLECOLUMN BOOK_COVER;

This statement creates the auxiliary table BLOB_AUX_TABLE_1 in the LOB table space created in 3.2.3, “Creating the LOB table spaces” on page 22. The other auxiliary table storing the CLOB column is created in a similar way.

There is no need to specify column names or column types for auxiliary tables. Using the STORES clause, new with Version 6, DB2 is told what column of which base table you want to store in the created auxiliary table. The associated table consists of only one column, the LOB column.

If the referenced base table is partitioned, there must be a LOB table space and an auxiliary table for each LOB column in each partition of the base table. In this case, Example 3-4 shows sample DDL for creating the auxiliary table.

Example 3-4 DDL for an auxiliary table containing data of one base table partition

CREATE AUXILIARY TABLE BLOB_AUX_TABLE_1 IN LOBDB.BLOBATS1 STORES BOOK_BASE_TABLECOLUMN BOOK_COVERPART n;

The PART clause indicates which partition’s BOOK_COVER column you want to store in this auxiliary table, where n is the number of the partition.

FIELDPROCs, EDITPROCs, VALIDPROCs, and check constraints cannot be defined on LOB columns.

How DB2 locates the assigned LOB valuesWe have only defined one column for the auxiliary table, but DB2 requires two more columns for the table: a column containing the ROWIDs of the base table and one column storing the current version number of the LOB. We report the information stored in the catalog for the three columns of the auxiliary table in Figure 3-9. Using this redundant data, DB2 is able to quickly locate rows in the auxiliary table. In order to do so, we need to define another and last new object: the auxiliary index.

Note: A LOB table space and its associated base table space must be stored in the same database. Otherwise SQLCODE -764 will be issued.


Figure 3-9 SYSIBM.SYSCOLUMNS contents for TBNAME= BOOK_AUX_TABLE

3.2.5 The auxiliary indexThe last step when creating LOBs is to define an auxiliary index for the LOB table space, as shown in Example 3-5. An auxiliary table must have exactly one index, and there cannot be any additional column in the index.

Example 3-5 DDL for an auxiliary Index

CREATE UNIQUE INDEX LOBDB.BLOBAIX1ON BLOB_AUX_TABLE_1USING STOGROUP BLOBSGIX

PRIQTY 100SECQTY 10;

As you can see, no index columns are defined within this index, because DB2 automatically creates the key definition. The index definition consists of a two key value: The 17-byte system generated ROWID stored as VARCHAR, and a 2-byte version of the LOB stored as SMALLINT, for 21 bytes in total (including the 2-byte length field for VARCHAR columns). No index keys can be defined. If you even try to specify a key column, DB2 will issue SQLCODE -767, missing or invalid column specification for an index. Figure 3-10 shows the key columns of an auxiliary index.

Figure 3-10 Index keys for an auxiliary index

Catalog descrip tion

LENG TH2 = 1 ,048,576

A UXID AU XVALU E AU XVER

VAR CH AR (17) BLO B (4) SM A LLIN T (2)

AUXID VARCHAR (17)

AUXVER SMALLINT

Auxiliary IndexAuxiliary Index


DB2 V7 uses the index on the auxiliary table to locate a LOB value for a particular row containing a LOB within the base table.

An index defined on an auxiliary table will automatically be defined as a unique index, fed by a ROWID from the base table. The buffer pool you may want to assign to this index does not need any special considerations, because the index only contains two relatively small columns.

Displaying LOB objectsFor our base table containing one BLOB column and one CLOB column, the display database will look like Example 3-6.

Example 3-6 Displaying a database for LOBs

DSNT360I -DB2G *********************************** DSNT361I -DB2G * DISPLAY DATABASE SUMMARY * GLOBAL DSNT360I -DB2G *********************************** DSNT362I -DB2G DATABASE = LOBDB STATUS = RW DBD LENGTH = 4028 DSNT397I -DB2G NAME TYPE PART STATUS PHYERRLO PHYERRHI CATALOG PIECE-------- ---- ---- ------------------ -------- -------- -------- -----BASETS TS RW BLOBATS1 LS RW CLOBATS1 LS RW BLOBAIX1 IX RW CLOBAIX1 IX RW ******* DISPLAY OF DATABASE LOBDB ENDED **********************

3.2.6 Adding a LOB column to an existing tableAnother possible situation is the need for adding a particular LOB column to an already existing table; this table becomes the base table for your LOB column(s).

Before a LOB column can be added, we have to be sure that a column of data type ROWID is already in the table. If it is not, we have to add it using the statement reported in Example 3-7.

Example 3-7 Adding a ROWID column

ALTER TABLE NEW_LOB_TABLEADD ROW_ID ROWID NOT NULL GENERATED ALWAYS;

A table can only have one ROWID column and you cannot add a ROWID column to a created temporary table.

We recommend that ROWIDs are always generated by using DB2’s mechanism. Specifying GENERATED BY DEFAULT generates a ROWID only if no value for the ROWID column is provided while inserting into the table. If a value for a ROWID column is provided, DB2 takes it, and inserts the value into the base table. By providing a value for a ROWID column, for instance when moving data across two DB2 subsystems, it is unlikely, but it may happen, that

Note: You cannot define an index on a LOB column.


DB2 generates a ROWID which is already stored in the table. Because the GENERATED BY DEFAULT clause always requires a unique index on that column, an insert with a DB2 generated ROWID value can result in SQLCODE -803. In this case it is your responsibility to resolve the duplication.

Once a column of data type ROWID is added, you can proceed with creating the LOB column(s) using the alter table statement shown in Example 3-8.

Example 3-8 Adding a LOB column

ALTER TABLE NEW_LOB_TABLEADD PICTURE BLOB(1M) NOT NULL WITH DEFAULT;

Adding a LOB column is not allowed for created temporary tables. The same is true when adding ROWID columns.

Instead of specifying BLOB (1M), like in this example, you can use the ALTER to define any other possible LOB column type to DB2.

After you have added to the base table the designated LOB columns, you should continue with the same actions shown earlier starting with 3.2.4, “Creating the auxiliary tables” on page 28. Remember that only one ROWID column is needed, even if you want to add more than one LOB column to the base table.

3.3 The CURRENT RULES special registerThe content of the CURRENT RULES special register has impact on your DB2 subsystem. The default value for this parameter is set via the SQLRULES parameter in your BIND PLAN statements. When you do not specify SQLRULES in your BIND PLAN statement, the default is DB2. Possible values for the register are ‘STD’ (standard) and ‘DB2’. We describe the effects of this parameter on your environment so that you can choose the desired value for your needs.

The CURRENT RULES register influences the following activities within your system:

� CREATE and DROP the auxiliary objects as LOB table space, auxiliary table, and auxiliary index

� Application development for LOBs using CURSORs

3.3.1 Impact on CREATE and DROP LOBsIn this section we examine the impact at CREATE and DROP time depending on the CURRENT RULES setting. You have to decide if you want DB2 to take care of your naming conventions, or if you prefer naming the objects and specifying the parameters by yourself. We recommend using CURRENT RULES DB2, since this enables you to set up the LOB environment more appropriate to your needs.

CREATE necessary LOB objectsIf you specify CURRENT RULES DB2, you have to create all required objects by yourself. You can then specify all the parameters mentioned in 3.2.3, “Creating the LOB table spaces” on page 22, and in the following sections.

Note: When you add one or more LOB columns to the base table, the table is marked incomplete until all dependent objects are created.


If you specify STD, DB2 creates all the auxiliary objects you need for the base table at the time it processes the CREATE statement for your base table. So DB2 creates the LOB table space, the auxiliary table and the auxiliary index for you. If the auxiliary objects defined in A.1, “DDL for a LOB environment” on page 138 had been created using CURRENT RULES STD, they would have looked as in Table 3-5 for the LOB table space, and in Table 3-6 for the auxiliary index.

Table 3-5 LOB table space created using CURRENT RULES STD

The name of the LOB table space is an 8-character string every time DB2 creates it for you. The first character is an L, followed by seven random characters. The auxiliary table is created as PAOLO.CLOB_DOCUM1LXA0LID. The table name of the auxiliary table is an 18-character string. The first five characters of the name are the first five characters of the name of the base table. The second five characters are the first five characters of the name of the LOB column. The last eight characters are randomly generated. If a base table name or a LOB column name is less than five characters, DB2 adds underscore characters to the name to pad it to a length of five characters.

Table 3-6 Auxiliary index created using CURRENT RULES STD

The name of the auxiliary index is also 18 characters long. The first character of the name is an ‘I’. The next ten characters are the first ten characters of the name of the auxiliary table. The last seven characters are generated randomly. The index has the COPY NO attribute, therefore full image copies and recover utilities are not allowed.

You can also modify an existing table for holding a LOB column using the ADD column option in the ALTER TABLE statement. First you ADD the ROWID column to your designated base table, then, at the time you ADD the appropriate LOB column with the ADD LOB column statement, DB2 also creates all the needed auxiliary objects, if CURRENT RULES STD is chosen. If CURRENT RULES DB2 is chosen, you have to create the objects on your own.

DROP base objectWhenever you drop a base table or a base table space, the associated objects are dropped depending on the way they were created. If you have created the auxiliary objects using CURRENT RULES STD, the LOB table space is implicitly dropped when you drop the base table or the base table space. The auxiliary table and the auxiliary index are dropped as well.

If you have created the auxiliary objects by yourself, only the auxiliary table and the auxiliary index are dropped when you either drop the base table space or the base table. The LOB table space remains in your system.

TSNAME BUFFERPOOL LOCKSIZE LOG CLOSE

L1LX$XO8 BP32K ANY YES YES

DSSIZE PQTY/SQTY FREEPAGE PCTFREE GBPCACHE

4G 12,800 / 12,800 0 5 NONE

IXNAME IXSPACE BUFFERPOOL CLOSE

ICLOB_DOCUM1LXARNX

ICLOBRDO BP2 YES

PIECESIZE PQTY/SQTY FREEPAGE PCTFREE

4,194,304 12 / 12 0 10


3.3.2 Impact on cursors fetching LOB valuesIn general, an application program declares a cursor and fetches the cursor in a specific section of an application program. The type of variables you fetch the cursor into cannot vary, even if you fetch the same cursor in two different sections of your application program for whatever reasons. Fetching a LOB column now gives you the flexibility to fetch into a large object host variable or into a large object locator variable. DB2 allows you to make a decision of the kind of variable you want to fetch the LOB value into, depending on the current value of special register CURRENT RULES at the time the cursor is opened.

Using CURRENT RULES DB2� After the cursor is opened, if the first FETCH executed uses a LOB locator to access a

LOB column, no subsequent FETCH for that cursor can fetch that LOB column into a LOB host variable.

� After the cursor is opened, if the first FETCH executed uses a LOB host variable to access a LOB column, no subsequent FETCH for that cursor can fetch that LOB column into a LOB locator.

Using CURRENT RULES STD� After the cursor is opened, and after the first FETCH executed, regardless of where the

LOB value is fetched into (LOB locator or LOB host variable), a subsequent FETCH for that cursor can FETCH the LOB column into either a LOB locator or a LOB host variable.

Although a CURRENT RULES special register set to STD gives you more flexibility to let your application program switch between both methods, you can get better performance if you use a value of DB2 in distributed environments. When you use the STD option, the server has to send and receive network messages for each FETCH to indicate whether the data being transferred is a LOB locator or a LOB value. With the DB2 option, the server knows the size of the LOB data after the first FETCH, so an extra message about the LOB data size is unnecessary. The server can send multiple blocks of data to the requester at one time, which reduces the total time for data transfer.

Using the STD option, DB2 cannot make any assumptions about what the requesting application may want on the next fetch.

3.4 Feeding a LOB columnCreating the environment for storing LOB values is one thing, providing the values is another. There are at least four different possible locations where the large objects are originating before they are inserted into a LOB column. The choice on how to bring them up to your host environment depends on their location and their size. Even if you have lots of large files already stored in your middleware, you can also consider uploading them via FTP to your host, and then access them with applications running on the mainframe. Applications running in a distributed environment can use the same technique as OS/390 applications for inserting LOBs, using distributed relational database architecture (DRDA) connections (for instance via DB2 Connect), open database connection (ODBC) calls, or Java using SQLJ or JDBC. Table 3-7 summarizes the alternatives on where LOBs can come from and how to feed them into your LOB columns.

Table 3-7 Where large objects come from and how

LOB size Where is the data? Method to feed LOBs

≤ 32 KB Client Cross-Loader, Application


3.4.1 LOAD with LOB columns smaller than 32 KBThis is probably the easiest way to bring your LOBs into DB2. Objects smaller than 32 KB are not really large, they may also be called small large objects, or SLOBs. For this kind of objects, consider also the alternative of storing them in a VARCHAR column, because LOB processing is more effective for large columns than for small columns. See Figure 7-7 on page 130 and Figure 7-8 on page 132 for a performance comparison between a VARCHAR column and an appropriate LOB column.

We also provide trace information on accessing a 32 KB column defined as VARCHAR and as a LOB column in 7.3.4, “Comparing SQL accounting profiles” on page 131.

If you are going to use the LOAD utility to load a base table, consider that the maximum row size of a row being loaded cannot exceed 32 KB. Every LOB data type (BLOB, CLOB and DBCLOB) in the input data set has to be preceded by its length in a 4-byte binary field. The length field does not include itself, only the length of the LOB data to be loaded. The length field must start at the column given as start-position in the LOAD statement.

For CLOB columns you can also specify MIXED when it contains mixed SBCS and DBCS data. If you specify MIXED, then any required CCSID conversions use the mixed CCSID for the input data, otherwise any conversions will use the SBCS CCSID for the input data. For further information regarding data conversions, see 3.1, “Data conversion” on page 14.

The LOAD statement may look like the one reported in Example 3-9 for loading a base table containing a simple CLOB column.

Example 3-9 LOAD a base table containing a LOB columns

LOAD DATA INDDN (SYSREC) RESUME YES LOG NO INTO TABLE PAOLO.CLOB_BASE_TABLE ( BOOK_NO POSITION (01:10) CHAR (10)

,DESCRIPTION POSITION (11:32) CHAR (32),BOOK_TEXT POSITION (43) CLOB )

If your base table is partitioned, you need to add the PART n clause to the LOAD statement. For more information on the LOG clause, see 6.2.1, “LOAD” on page 94 and specifically Table 6-1 on page 95.

Since the maximum record length of a data set is 32,760, this is not a very likely way to feed your LOBs in the database, especially if there is more than one LOB column in your base table.

3.4.2 Inserting LOBs via host applicationThe LOAD utility cannot feed our LOB columns with values greater than 32 KB. To handle this situation, and assuming that the LOB data is already available in your OS/390 environment, you have to write an application program which performs the inserts of your LOBs.

≤ 32 KB Host LOAD, Application

> 32 KB Client Application, DB2 Extenders

> 32 KB Host Application

LOB size Where is the data? Method to feed LOBs


You have to distinguish between two situations:

� Inserting LOBs using a host variable large enough to hold the entire LOB value

� Inserting LOBs using a host variable not large enough to hold the entire LOB value, and introducing the use of locators

Inserting LOBs using a host variableInserting a LOB already available at the host is probably the second way to feed your LOBs. There are two main techniques on how to move a LOB value to your host variable. The first one requires a data set containing the entire LOB value in one row. The only thing the application has to do is read a file, move the value to the assigned host variable, provide the correct value to the length field, and execute the SQL INSERT statement. The second technique consists of having a data set where a certain number of rows have to be tied together to build the entire LOB. This may be a more common method because of the maximum MVS allowed record length of 32,760 bytes, which is really a small LOB. The rows can be tied together in one variable as shown in the pseudo-code in Example 3-10.

Example 3-10 Tieing rows together in one variable

MOVE 0 TO HV-LOB-LENGTHPERFORM UNTIL EOF-INPUTREAD FILE INTO INPUTRECORD

IF NOT EOF-INPUTMOVE LENGTH OF INPUTRECORD TO LENGTHMOVE INPUTRECORD TO HV-LOB-DATA (HV-LOB-LENGTH + 1:LENGTH)ADD LENGTH TO HV-LOB-LENGTH

END-IFEND-PERFORM

The definition of a host variable, large enough to contain an entire LOB value, is reported in Example 3-11. You need to include it in your WORKING-STORAGE SECTION of your application program if you try to insert a 1 MB CLOB, as in our example.

Example 3-11 host variable declaration for a LOB column

01 HV-LOB USAGE IS SQL TYPEIS CLOB (1M).

Using this syntax, DB2 generates the definition reported in Example 3-12 for your application program.

Example 3-12 What the DB2 precompiler makes of your host variable declaration

01 HV-LOB. 02 HV-LOB-LENGTH PIC S9(9) COMP. 02 HV-LOB-DATA. 49 FILLER PIC X(32767).

[repeated 31 times] 49 FILLER PIC X(32).

The last filler is used to match exactly the requested host variable size of 1 MB as declared for the CLOB. Be aware that your application only defines host variables in a size you are allowed to acquire, regarding your JCL and your system. You must acquire buffers large enough to store your LOBs. This may be difficult for very large LOBs. For an example, IBM COBOL for OS/390 and VM 2.2.1 supports up to 16,777,215 bytes (16 MB minus one byte) for one item in the WORKING-STORAGE SECTION or in the LINKAGE-SECTION. Because


of this limitation, for LOBs bigger than 16 MB minus 1 byte, you have to use another method for feeding your LOBs. See “Inserting LOBs using locators” on page 36” for details. The application has to ensure the delivery of a correct value in the length field of your LOB host variable.

Once the host variable contains the entire LOB, you can simply issue an SQL INSERT on the base table to pass your LOB to DB2. You do not need to worry about storing LOBs in the auxiliary table — this is DB2’s job. From the application programmer’s point of view there is only one table containing all the columns being used for LOB processing. The statement shown in Example 3-13 is a possible way for inserting your LOB data.

Example 3-13 Inserting a single LOB value using one host variable

EXEC SQL INSERT INTO BASE_TABLE (COL1, COL2, LOB) VALUES (:HV-COL1 ,:HV-COL2 ,:HV-LOB)END-EXEC

It is recommended that an application commits after completing the unit of work while inserting a LOB, because COMMIT releases X-LOB locks taken during insert and frees allocated storage in the data space used by the LOB. A sample program showing how to insert a LOB using one host variable is included in the SAMPLE1.TXT file described in Appendix D, “Additional material” on page 155. You can find more information on inserting LOB values and DB2 data spaces in “INSERT” on page 122.

This method is one of the fastest to feed your LOBs, because SQL does not have to be invoked many times. The only restriction is that inserting LOBs bigger than the maximum supported variable size of your programming language is not possible using this method. If you want to insert a LOB larger than the maximum supported variable size of your application language, please refer to the section “Inserting LOBs using locators” on page 36.

Inserting LOBs using locatorsIf you are not able to acquire a buffer large enough to hold your LOB data, you have to move it in pieces into a DB2 LOB column. This technique is supported by using LOB locators. We show two examples of pseudo code using locators. In the first one we append values in single chain passing through an intermediate locator till we are ready to insert the full LOB, in the second one we use instead multiple chains of locators.

Single locator chainFor this case see the pseudo-coding reported in Example 3-14 for a suggestion on inserting large LOBs.

Example 3-14 Pseudo-code inserting LOBs > 32 KB with one locator chain

Definitions:

HV-LOB USAGE IS SQL TYPE IS CLOB (1M)LOB-LOCATOR-1 USAGE IS SQL TYPE IS CLOB-LOCATORLOB-LOCATOR-2 USAGE IS SQL TYPE IS CLOB-LOCATOR

Pseudo-Code:

MOVE 0 TO HV-LOB-LENGTHEXEC SQLSET :LOB-LOCATOR-1 = ‘’

END-EXEC


PERFORM UNTIL EOF-INPUTREAD FILE INTO INPUTRECORD

IF NOT EOF-INPUTPERFORM BUILD-HOST-VARIABLE

END-IFEND-DO

PERFORM FINAL-INSERT

:BUILD-HOST-VARIABLEMOVE LENGTH OF INPUTRECORD TO LENGTHMOVE INPUTRECORD TO HV-LOB-DATA (HV-LOB-LENGTH + 1:LENGTH)

ADD LENGTH TO HV-LOB-LENGTH

IF HV-LOB-LENGTH > 1000000 THENPERFORM APPEND-LOCATOR

MOVE 0 TO HV-LOB-LENGTHMOVE SPACE TO HV-LOB-DATA

END-IF

:APPEND-LOCATOREXEC SQLSET :LOB-LOCATOR-2 = CONCAT (:LOB-LOCATOR-1, :HV-LOB)

END-EXEC

EXEC SQLFREE LOCATOR :LOB-LOCATOR-1

END-EXEC

MOVE LOB-LOCATOR-2 TO LOB-LOCATOR-1

:FINAL-INSERTIF HV-LOB-LENGTH > 0 THENPERFORM APPEND-LOCATOR

END-IF

EXEC SQLINSERT INTO BASE_TABLE(KEYCOL1, COL2, LOB)

VALUES(:HV-KEYCOL1, :HV-COL2, :LOB-LOCATOR-1)

END-EXEC


END-EXEC

At the top of Example 3-14 you can find the declaration of a CLOB host variable. In this case the size is 1 MB because we assume 1 MB as the maximum usable host variable size in our environment. The associated locators are also defined at the beginning of the pseudo-code.

The first initialization of LOB-LOCATOR-1 is done by its first reference in the CONCAT statement. Then the application reads the input file in a loop and it builds a host variable using the rows from the input file until an end-of-file condition occurs. After reading one input record, it is appended to our 1 MB CLOB host variable. After the host variable is nearly filled up (CURRENT-SIZE > 1000000, the correct value depends on the possible length of your


input records), the host variable is appended to LOB-LOCATOR-1 using the CONCAT function of DB2 to the already existing LOB value, where LOB-LOCATOR-1 is referring to, and set to LOB-LOCATOR-2. The locator is not assigned to a specific table yet, only the reference to a value inside of DB2 is created. Using this technique, LOB materialization takes place outside of the application in the data space created dynamically by DB2 for building the entire LOB.

If the host variable is filled with some data even after an end-of-file condition was detected, the content of the host variable is applied one last time to the locator. After all records are appended to your LOB locator (ensure that the correct record length is passed to DB2), the application finally inserts the LOB, using the locator for providing the needed host variable for the LOB value.

Attention should be paid to the FREE LOCATOR statement. If you do not FREE the used locators, DB2 will tend to keep them around in buffers allocated out of the DBM1 address space where the locators reside, and this may cause problems in already virtual storage constrained environments. The intent of the FREE LOCATOR statement is to release the allocated virtual storage space used by the locator itself as well as the virtual storage buffers in the data space containing the data referenced by the locator. Unless the LOCATOR has been defined with HOLD, an SQLCOMMIT will also free the locator as well as the related and allocated entry in LOB data space. If you do not free the locators after using them, the storage will remain allocated until COMMIT. If you are trying to insert really large LOBs, your application could fail for having reached the limit of virtual storage set for the allocated data space, or the DBM1 address space could reach its limit because of heavy usage of storage caused by the storage structures associated with active locators.

Make sure to issue the FREE LOCATOR statement for each append within the concatenation loop as well, this decreases an internal counter used by DB2 to control the structure built to reference the first locator. This minimizes the risk of causing virtual storage problems when inserting many LOBs without issuing a COMMIT between your LOB insert statements. We recommend for you to COMMIT after each single LOB is inserted by your application. The MOVE simply moves the value of our second locator to the first locator, which makes the first locator available again but with the value of the second locator which we use temporarily for the concatenation.

We have used a final insert statement at the end of the sample program, because we want to avoid long lock durations on the base table. Another technique would be to insert the first input record into the table, assign a locator to the LOB value and start building the locator for a final update on the LOB column using the locator. When you plan to use the updating method, be aware that you hold a lock on the base table and on the auxiliary table. The exclusive lock on the base table probably prevents access by other users to the base table, depending on the lock size you use.

Error-handling routines and possible data movement to non-file-variables are not mentioned in the example above.

A sample program showing how to insert a LOB using a 10 MB host variable and a single locator chain is included in the SAMPLE2.TXT file described in Appendix D, “Additional material” on page 155.

Multiple locator chainsThe method described in Single locator chain is fast, but there is also one disadvantage while using it: Every pass through the loop acquires a new LOB locator also pointing to a previous one. Using this method, DB2 builds a chain of locators. Each locator is associated with a control structure that describes how to construct the new LOB value by concatenating a value copied from HV-LOB-DATA into a LOB data space with the contents of the identified LOB


locator. This particular use of LOB locators does not hurt in terms of additional storage acquired in the LOB data space, the agent’s LOB storage limit and the system’s storage limit. But it is not too hard to imagine situations where you could run out of space very quickly without proper locator management.

Each time a concatenation is added to the chain, the entire chain is interrogated via recursion. Let us consider the chains we build using the concatenation technique as secondary chains. See Figure 3-11 for a better understanding of what happens in a data space when you use the appending mechanism as mentioned above.

Figure 3-11 Secondary chain of locators

After assigning the first version of a locator to a host variable, the locator only contains the first host variable (Version 1). After issuing the first concatenation of :LOCATOR-1 = CONCAT (:LOCATOR-1, :HV-LOB), the second version of the same locator is created. Version 2 of the locator contains the new host variable and points to Version 1 of the same locator. So DB2 builds a chain of locators. Appending data to a locator becomes more expensive when the chain grows, because of the recursive interrogation technique we mentioned above. You can especially run into a long chain when you build a large object of comparatively small input records only using locators. This is not the recommended way to feed LOBs, but you may probably run into this situation.

To improve performance, we can build a primary chain containing the previously built secondary chain to reduce the number of recursions when appending a new host variable to LOCATOR-1. So DB2 only has to go through the long chain containing all host variables when we append a secondary chain to a primary chain. See Figure 3-12 for a visualization of primary and secondary chains.

host variable# 3

host variable# 2

host variable# 1

LOCATOR-1Version 3

LOCATOR-1Version 2

LOCATOR-1Version 1


Figure 3-12 Primary chain of locators containing secondary chains

As soon as a secondary chain becomes very large, we can source it out to a primary chain to reduce the level of recursion we have to go through when we append another host variable to our secondary chain. You can use this method, when appending big host variables to a single locator does not satisfy your performance requirements after a large number of concatenations for a secondary chain.

The pseudo-code reported in Example 3-15 shows you how to perform the same insert as above including a technique dealing with primary and secondary chains.

Example 3-15 Pseudo-code inserting LOBs > 32 KB with multiple locator chains

Definitions:

HV-LOB USAGE IS SQL TYPE IS CLOB (100K)LOB-LOCATOR-1 USAGE IS SQL TYPE IS CLOB-LOCATORLOB-LOCATOR-2 USAGE IS SQL TYPE IS CLOB-LOCATOR

Additional instruction before reading the 1st INPUTRECORD:EXEC SQLSET :LOCATOR-2 = ‘’

END-EXEC

Pseudo-Code:

:APPEND-LOCATORADD 1 TO APPEND-LOCATOR-COUNTER

EXEC SQLSET :LOB-LOCATOR-1 = CONCAT (:LOB-LOCATOR-1, :HV-LOB)

END-EXEC

IF APPEND-LOCATOR-COUNTER = 1000 THENEXEC SQLSET :LOB-LOCATOR-2 = CONCAT (:LOB-LOCATOR-2, :LOB-LOCATOR-1)

END-EXEC

host variable# 3

LOCATOR-1Version 3

host variable# 6

LOCATOR-1Version 6

host variable# 2

LOCATOR-1Version 2

host variable# 5

LOCATOR-1Version 5

host variable# 1

LOCATOR-1Version 1

host variable# 4

LOCATOR-1Version 4

LOCATOR-2Version 1

LOCATOR-2Version 1

LOCATOR-2Version 2


EXEC SQLSET :LOB-LOCATOR-1 = ‘’

END-EXEC

MOVE 0 TO APPEND-LOCATOR-COUNTEREND-IF

:FINAL-INSERTIF HV-LOB-LENGTH > 0 THENPERFORM APPEND-LOCATOR

END-IF

IF APPEND-LOCATOR-COUNTER > 0 THENEXEC SQLSET :LOB-LOCATOR-2 = CONCAT (:LOB-LOCATOR-2, :LOB-LOCATOR-1)

END-EXECEND-IF

EXEC SQLINSERT INTO BASE_TABLE(KEYCOL1, COL2, LOB)

VALUES(:HV-KEYCOL1, :HV-COL2, :HV-LOB-LOCATOR-1)

END-EXEC

EXEC SQLFREE LOCATOR :LOB-LOCATOR-1,:LOB-LOCATOR-2

END-EXEC

The difference between these two examples is the locator management in the APPEND-LOCATOR subroutine. After 1,000 times of appending a host variable (which means nearly 1 GB of data in our example), we create a primary chain by issuing SET :LOCATOR-2 = CONCAT (:LOCATOR-2, :LOCATOR-1). After LOCATOR-1 was outsourced successfully, we can reset it to point to an empty string to start building a new secondary chain.

It is good programming practice to free both locators after the final insert statement in order to release the virtual storage allocated out of the data space, and make it available for further usage in the application. Otherwise the storage remains allocated until the implicit or explicit COMMIT releases the entire data space.

Before you start coding your application, consider that every invocation of SQL takes a certain amount of time. So try to keep SQL calls at a minimum number and exploit as much storage as you can get for your host variable. This results in a reduction of SQL calls and leads you to a better performing application. But also be aware of storage allocations issues if you do not free your locators.

A sample program showing how to insert a LOB using a 10 MB host variable and two locator chains is included in the SAMPLE3.TXT file described in Appendix D, “Additional material” on page 155.


Chapter 4. Using LOBs

In this chapter we discuss topics related to the differences in using LOBs when compared with the use of the normal DB2 objects. The chapter is structured as follows:

� SQL semantics with LOBs� LOB locators� Handling BLOBs, CLOBs, DBCLOBs� LOBs are different DB2 objects� Locking� Details about ROWID

4


4.1 SQL semantics with LOBsThere are some differences between SQL functions when accessing LOBs and when accessing other normal columns stored in a table. Logically, a row in the base table also contains the LOB value, and this is true from the application’s point of view. Physically, DB2 stores them in two different table spaces. You cannot access the auxiliary table via SQL where DB2 stores the LOB values. DB2 protects the auxiliary table by issuing SQLCODE -766 when you try to access it directly by using SQL.

In general, LOBs can be referenced in all the string functions with the exception of those that relate to date and time. LOBs have the same set of restrictions as other long strings. Some functions cannot be used on LOB columns for obvious reasons. You cannot use the following functions on LOB columns:

� GROUP BY clause� HAVING clause� ORDER BY clause� SELECT DISTINCT clause� Column function� Datetime function� DECIMAL or NULLIF function� WHEN clause of a CASE expression� Subselect of a UNION without the ALL keyword

Most of the restrictions on LOB values are due to the fact that LOB values cannot be compared, except with the LIKE predicate.

LOB functionsOne way of manipulating large objects without retrieving the entire object is to use functions. Many of the string functions also work with LOBs. The built-in functions allow you to concatenate strings, get a substring, find the LOB length, find the position in the LOB of a search string, and cast the LOB to another type. UDFs can be used as well.

The list of available LOB functions includes:

� CONCAT� SUBSTR� LENGTH� POSSTR� IFNULL� VALUE / COALESCE� Casts� UDFs� LIKE within predicates

In the following sections we discuss the most important of the functions listed above.

How CONCAT operatesThe CONCAT function allows you to put two strings together so that they end up as one string. You can do the same with your host language, but depending on what language you use there are some limits within which you cannot move easily. For example, you may not be able to concatenate two host variables containing each 10 MB of data, because the resulting variable would be bigger than the maximum allowed size of a variable, as allowed by COBOL for OS/390 and VM 2.2.1. But be aware not to use the CONCAT function every time you want to string two short variables together (like two variables of each 80 bytes), because invocation


of SQL is more expensive than a single MOVE statement in COBOL, for example. Try to avoid unnecessary use of this function and use simple statements in your host-language as long as you can afford it. You can spend much CPU and elapsed time on thoughtless use of CONCAT operations.

An SQL statement without any data access like this one uses about 7,500 machine instructions more than the same MOVE statement without any SQL invocation. The amount of data being moved is nearly the same, so you can assume the mentioned overhead for concatenating variables using SQL compared to a MOVE statement.

A few details about SUBSTRYour application programs can use the SUBSTR function to retrieve part of a LOB value. The string delivered to your application can consist of up to 2,147,483,647 bytes, so there are no limits in accessing parts of your LOBs. When your declared LOB column is smaller than 2 GB, the maximum value returned by the SUBSTR function is the declared size of your large object. Depending on what SUBSTR operation you perform, DB2 does not materialize the LOB, because the information where to look for the requested substring is stored in the LOB map pages.

The built-in function POSSTRYou can use the POSSTR function, new with DB2 V6, to locate the starting position of one string within another string. POSSTR returns the first occurrence of one string. The string you want to locate (the search string) in the source string can consist of up to 4,000 bytes, which can also be represented by a host variable. The POSSTR function returns the position of your search string for BLOB and CLOB values. For DBCLOBs, a returned position is a DBCS character. For finding the second, third or nth occurrence of a string using POSSTR, please refer to 4.3.3, “Finding the nth occurrence of a string” on page 57.

IFNULL, VALUE, and COALESCEWhen you use the IFNULL function in your select statement, you can tell DB2 which value you want it to return to your application when an accessed LOB value is null. For example, you can return text saying ‘value unknown’ to your application if a particular LOB value is null. You can also return the value of an already assigned LOB locator, probably pointing to a picture saying “Image not found”.

IFNULL (:LOB-LOCATOR, ‘unknown value’)IFNULL (:LOB-LCATOR, :LOCATOR-IMAGE-NOT-FOUND)

IFNULL is identical to the COALESCE and VALUE functions, except that IFNULL is limited to two arguments instead of multiple arguments. Because a NULL indicator is one of the two stored flags in the base table, DB2 does not need to access the auxiliary table at all, if a LOB column is stored as a null value.

Using CAST for LOBsYou can also use CAST functions on your LOB values or LOB locators, even to convert your current LOB value into another value. They can be used to get around some of the restrictions on LOB values. To give you an idea of CASTing between data types, assume the following statement:

SELECT LOB FROM BASE_TABLEWHERE SUBSTR (LOB,1,11) = ‘IBM Redbook’

You could expect that the statement returns you all the LOB values which start with ‘IBM Redbook’. Because large objects are subject to long string column restrictions, DB2 issues SQLCODE -134, complaining about an improper use of a long string column. To solve this problem, you can replace the statement using the following syntax:

Chapter 4. Using LOBs 45

SELECT LOB FROM BASE_TABLEWHERE CHAR (SUBSTR (LOB,1,11)) = ‘IBM Redbook’

This statement delivers the result you could have expected when you issued the first statement.

Table 4-1 shows the list of the allowed LOB conversions.

Table 4-1 Casting large objects

The reason for the length growing up to three times with DBCLOB conversions into a CHAR, VARCHAR or CLOB column, is to allow for expansion when the data is converted from UTF-16 to UTF-8. A character that takes two bytes to represent in UTF-16, can take three bytes in UTF-8.

How does LIKE operate on LOBs?Depending on the value specified for LIKE search arguments, LOB Manager takes different actions. These actions correspond to common LIKE operations used for non LOB values. When you issue LIKE ‘Chapter 8%’, only the first nine bytes of a LOB value according to qualifying base table rows are scanned to verify the result. A LIKE ‘%Chapter 8%’ clause can be worse, because now DB2 has to scan the entire LOB value until it finds the first occurrence of the string to qualify the LOB value for your SQL statement. This kind of statement can be very time consuming, depending on your average LOB size, and the number of touched rows in the base table.

In general, after issuing an SQL statement against the base table, all base table columns appearing in the WHERE clause of your statement are checked before a condition on the auxiliary table is checked, even if the columns in the base table are not indexed. This avoids unnecessary scans of your LOB values and only those LOB values are scanned which already have qualified from the base table point of view.

4.2 LOB locatorsYou can use locators everywhere in your application program where you can access a LOB value by itself. The idea behind locators is that an application program only deals with a reference to a particular LOB and DB2 performs the real operations on the LOB value. Using this method, the LOB value is not stored in the application’s memory. To deal with a locator instead of the entire value, an application program simply selects the LOB column into a LOB locator host variable, not into the generated LOB host variable. By selecting into a locator variable DB2 assigns a 4-byte value to the locator which is delivered to the application. The entire LOB value is not retrieved nor delivered to the application. By selecting the whole LOB into a locator variable, you avoid materialization of the LOB, because DB2 does not have to access every page of the LOB, it just keeps the reference.

BLOB CLOB DBCLOB

CHARVARCHARCLOBGRAPHICVARGRAPHICDBCLOBBLOB

CHARVARCHARCLOBGRAPHIC (*)VARGRAPHIC (*)DBCLOB (*)

CHAR (**)VARCHAR (**)CLOB (**)GRAPHICVARGRAPHICDBCLOB

(*) CAST is only supported if the data is Unicode.(**) CAST is only supported if the data is Unicode. The result length for these casts is 3 * LENGTH (graphic string)


For example, when selecting a LOB value, an application program could select the entire LOB value and place it into a host variable of the same size as the LOB (which is acceptable if the application program is going to process the entire LOB value at once), or it could instead select the LOB value into a LOB locator. Then, using the LOB locator, the application program can issue further database operations on the LOB value (such as applying scalar functions SUBSTR, CONCAT, LENGTH, doing an assignment, searching the LOB with LIKE or POSSTR, or applying UDFs against the LOB) by supplying the locator as input. The resulting output of the locator operation, for example the amount of data assigned to the application’s host variable, would then typically be only a subset of the input LOB value.

Once a locator is set to a particular LOB value, there is no action you can take within the data base to change that value from the application’s point of view until the locator is freed. You can free a locator by issuing a FREE LOCATOR statement or by completing the current unit of work.

But how does this work? Locators do not force extra copies of the data in order to provide this function. When you assign a locator to a specific LOB value, an S-LOB lock (see 4.5, “Locking” on page 62 for a description of LOB locks) is acquired on the LOB value, and it is only released at COMMIT or when you explicitly free the locator. When you issue the HOLD LOCATOR statement, an assigned locator survives the current unit of work and is valid until the thread terminates or a FREE LOCATOR statement is passed to DB2. Either one of these events will release the S-LOB lock taken on the LOB value in the auxiliary table.

4.2.1 Getting to know LOB locatorsYou assign a locator to a LOB value when selecting a LOB column into a LOB locator. Using this method, DB2 detects that the entire LOB is going to be selected into a locator column and therefore it does not materialize the associated LOB value and also does not provide the LOB value to your application program. Instead, DB2 provides a value for your locator into your locator host-variable where your application is selecting the LOB value into. You can use a LOB locator anywhere a LOB value can be used, for instance in expressions where you would normally use an entire LOB value.

Can the LOB value change while a locator is assigned?Simply speaking: Yes, it can. But it will not change for your application program once you have associated a locator to a particular LOB value. So what is DB2 doing when you have a locator assigned to a LOB value (remember: a locator represents a LOB value at a point in time) and somebody else is updating or deleting the LOB? DB2 ensures that the locator still represents the original value that existed at the time the locator was set. See figure Figure 4-1.


Figure 4-1 Concurrent LOB access using LOB locators

In this example, transaction 1 selects a non-LOB column into a host-variable and also a LOB column into a locator variable. After the SELECT is completed in transaction 1, transaction 2 updates the LOB column which is already referenced by a locator in transaction 1. After the LOB is updated by transaction 2, a new value is inserted into the VIDEOLIB table in transaction 1, using the previously assigned locator for the LOB value by DB2. But the value referenced to by the locator is still the same as before the updating transaction has updated the LOB value. How is this possible?

Operations on the original LOB value have definitely no effect on the value referenced by a locator.

Let us first have a look at LOB delete: a DELETE statement only de-allocates pages in the Auxiliary Table where the LOB data is stored. But the data still remains in the pages, it is not deleted.

Now let us have a close look on how a LOB is updated: Updating a LOB consists of DELETE and INSERT, so the pages are de-allocated again and the data still remains in the auxiliary table.

The INSERT statement cannot use the de-allocated pages because of the S-LOB lock on the LOB value acquired by the former issued SELECT statement into the LOB locator.

The conclusion is that these data pages where a locator is pointing to cannot be allocated again, even if they are going to be de-allocated by a DELETE statement, as long as an S-LOB lock persists on the referenced LOB value.

01 movie_loc ... IS SQL TYPE IS BLOB-LOCATOR

SELECT MOVLENGTH, LOBMOVIEINTO :length, :movie-locFROM LITERATUREWHERE TITLE = 'The Maltese Falcon';

INSERT INTO VIDEOLIB (LENGTH, FILM, DESC)VALUES(:length, :movie-loc, 'Uncut/Original');

UPDATE LITERATURESET LOBMOVIE=:edited-versionWHERE TITLE = 'The Maltese Falcon';

TRANSACTION 1

TRANSACTION 2


Using locators across multiple units of work A LOB locator is only a mechanism used to refer to a LOB value during a unit of work. Ordinarily, a locator is freed, implying release of acquired space by DB2 in a LOB data space if the locator is not referenced further, when the application COMMITs its data or the associated thread terminates. Acquired locks are also released at COMMIT time as usual. To increase locator durations beyond COMMIT points, a HOLD locator statement can be issued. This statement increases the duration of the specified locator, and the locks held on the locator, until a FREE locator statement is issued for the same locator. SQL ROLLBACK also frees every locator having the hold property. If no FREE locator statement is issued, the locator is valid until the thread terminates. In Example 4-1 you can find the syntax of FREE locator and HOLD LOCATOR statements.

Example 4-1 Syntax for FREE locator and HOLD locator

Hold beyond COMMIT:

EXEC SQLHOLD LOCATOR :HV-LOCATOR-1, :HV-LOCATOR-2

END-EXEC

Free the locator if it is not needed any more:

EXEC SQLFREE LOCATOR :HV-LOCATOR-1, :HV-LOCATOR-2

END-EXEC

A locator is freed when one of the following conditions occur:

� An SQL FREE LOCATOR statement occurs� An SQL ROLLBACK statement occurs� The associated thread terminates

If a locator has the hold property, it survives the SQL COMMIT statement. Without the hold property, it does not survive the SQL COMMIT statement. A locator obtains the hold property by the SQL HOLD LOCATOR statement.

If you receive SQLCODE -423 (INVALID LOCATOR VALUE) after you have specified more than one host-variable for a FREE LOCATOR statement, only those locators up to the invalid locator will be freed. According to this result, when you receive SQLCODE -423 after issuing a HOLD LOCATOR statement, all locators listed in the statement after the first invalid locator will not be held.

Can I free a referenced locator?Assume two locators as shown in Figure 3-12 on page 40, where LOCATOR-2 references LOCATOR-1. When you now free LOCATOR-1 using the FREE LOCATOR statement, it does not release any DB2 resources in this case, since LOCATOR-2 still refers to LOCATOR-1. The locator is only freed externally. This means that the application cannot refer to it; when it does, it probably gets SQLCODE -423 indicating an invalid locator. Internally, DB2 keeps LOCATOR-1 around as long as it is referenced.

When to use locatorsIf LOBs are not too large, they can be managed as other data. LOBs may be retrieved, inserted and updated, just like any other type of data. Everything works well as long as you have enough main storage space to store the LOB. If you think of a CLOB containing a book’s text and of an application which has to extract only a single chapter of the book, without the use of locators you have to retrieve the whole object (probably 2 GB of data) in your


application and then to extract the data you need. Because of their size, LOBs may be unmanageable for a single application. Huge amounts of storage may be needed to buffer their values, and it may not be possible to acquire contiguous buffers of sufficient size, since this storage must reside within the region size of the address space in which the application program is running. To give you an easy method to deal with LOB columns, DB2 provides locators to make LOB access manageable.

Using a locator may be a good choice in the following situations:

� Only a part of a LOB is needed by the application� Only a part of a LOB is moved to a client� The entire LOB does not fit into the application’s memory� A temporary LOB value is needed but it has not to be stored in DB2

You cannot use any of the locators mentioned above in mathematical operations. The reason is due to the need of preventing the locators from being corrupted by the application.

Materialization when using locatorsIf you do not use the locator technique to access your LOBs, then DB2 has to materialize the value represented by a large object when you access it. In this case, DB2 materializes the LOB by moving it through the buffer pool into the user’s address space. Basically, materialization is avoided when you select LOBs or parts of their values into locators. When an application retrieves a LOB value, whether it is assigned to a locator or not, it goes through the buffer pool into the user address space. As soon as a LOB value has to be materialized and a locator is involved, materialization of a LOB takes place in a data space created by DB2. This mainly occurs when you update a LOB value by using locators. In this case, the DBM1 address space only contains the LOB’s control structure. DB2 tries to avoid materialization wherever it is possible, but if it is no longer possible, it will materialize a certain value in a data space or in the user address space, depending on the way a LOB is accessed. For more information about materialization, see 7.1, “LOB materialization” on page 120.

Locators and expressionsLOB locators may also represent more than just base values. They can also represent the value associated with a LOB expression. For example, a LOB locator can represent the value associated with:

EXEC SQLSET LOCATOR = SUBSTR (LOB1, :START1, :LENGTH1) CONCAT

SUBSTR (LOB2, :START2, :LENGTH2)END-EXEC

The same statement can also be issued even if LOB1 and LOB2 are referenced by locators. You are also able to use other LOB functions for further reference in an expression. Regarding this implementation, DB2 for OS/390 and z/OS provides you with the opportunity to build new locators using other locators and expressions of LOBs or locators. This gives you the flexibility to deal with large objects in various scenarios without allocating huge amounts of storage when you deal with LOBs.

Examples of using locatorsA LOB locator is also allowed to represent LOB expressions, this means substrings of an entire LOB value. A LOB expression is defined as any expression that refers to a LOB column or results in a LOB data type. LOB functions may also be part of a LOB expression. Also LOB expressions may reference other LOB locators, which does not simplify this topic at all.


A LOB expression is any string expression that contains a LOB value. It is possible to associate the result of a LOB expression to a LOB locator. Since a LOB locator can be used anywhere a LOB value can be used, the LOB expression associated with a LOB locator could make reference to other LOB locators.

Concatenating two strings using locatorsWhen you plan to use any string expressions on a LOB value, you can also use a locator instead of the entire LOB value. For example: when you try to concatenate two LOBs of 10 MB each, you do not have to read them both into two large host variables and string them together, you can instead let DB2 do the work for you.

Assume that LOCATOR-1 is pointing to a CLOB value containing a 10 MB string, and LOCATOR-2 is pointing to another CLOB value of another 10 MB. After assigning both locators, you can simply issue this to string them together:

EXEC SQLSET :LOCATOR-3 = CONCAT (:LOCATOR-1, :LOCATOR-2)

END-EXEC

The new value of both strings, now a total 20 MB, is assigned to the new locator LOCATOR-3 where you can now refer to. To store the new value in a table, you can insert a LOB using the locator as a reference in an insert statement or perform other operations like SUBSTR.

Referring to a block of data inside of a CLOBIf you want to refer to a single chapter in a document, you can do this also by simply using locators. The first thing an application has to do is to assign a locator to a particular LOB value. The starting position of the chapter you want to refer to is easily determined by the integer value resulting from the POSSTR function as follows:

EXEC SQLSET :START-POSITION = POSSTR (:LOB-LOCATOR, ‘Chapter 8’)

END-EXEC

You can determine the end of the chapter you want to deal with in the same way using the POSSTR function. In our example the position of the beginning of Chapter 9:

EXEC SQLSET :END-POSITION = POSSTR (:LOB-LOCATOR, ‘Chapter 9’)

END-EXEC

After issuing this statement, the locator END-LOCATOR points to the beginning of Chapter 9. When your application wants to deal with Chapter 8 directly, you can set the content of the chapter to a host variable being large enough to contain the entire text for Chapter 8. Otherwise simply set the string containing Chapter 8 to a new LOB locator as shown below:

EXEC SQLSET :CHAPTER-8-LOCATOR = SUBSTR (:LOB-LOCATOR

,:START-POSITION,:END-POSITION - START-POSITION)

END-EXEC

The SUBSTR function uses the LOB locator (instead of the whole LOB value) as a string-expression, the START-POSITION represents a starting-position and the expression END-POSITION - START-POSITION is provided as a length value.

Note: Try to avoid the use of string expressions (such as SUBSTR) of BLOB values, because binary documents may not be usable in parts, such as pictures, executable files or even movies. The use of parts for further processing depends on the type of data you store in your LOB columns.


4.3 Handling BLOBs, CLOBs, DBCLOBsDealing with all kinds of large objects is challenging from the application’s point of view. You face new challenges in dealing with objects bigger than most common data objects. In this section we discuss application programming techniques to manipulate LOBs without retrieving them, and also unloading LOBs to store them in a data set. If you need your data in a flat file, you cannot use normal unloading techniques, because the data has to be split over more than one row (the maximum LRECL you can use is 32,760). You can find examples for inserting LOB data via an application in 3.4, “Feeding a LOB column” on page 33.

4.3.1 Manipulating a LOB without retrieving itIn most common scenarios you would only need to manipulate CLOBs. Manipulating BLOBs can result in unusable binary data if important parts are removed or updated. The updated binary value may become unreadable for the application using this data. As an example, just think of updating parts of a JPEG picture or an MPEG movie. Cutting out or updating some data in the movie may destroy it for further use.

Case 1: Deleting a specific part of a LOBTo delete a special part of a LOB value, the first step is to locate the start and the end positions for your delete. In our example we are going to delete ‘Chapter 8’ of our book CLOB. The first step our application should do is to assign a locator on the LOB we want to update. After the locator is set, we use the POSSTR statement to figure out the position of the beginning of ‘Chapter 8’. The position of the end of ‘Chapter 8’ in our book is marked by the string ‘Chapter 9’. When our application knows both positions, it can assign a new locator using the SUBSTR function to point to the beginning of the book up to the beginning of ‘Chapter 8’ minus one byte, concatenating the rest of the book beginning at the end position of ‘Chapter 8’, which is the beginning of ‘Chapter 9’. After the new locator is established, the LOB is finally updated. See Example 4-2 for a pseudo-code performing the actions as mentioned above.

Example 4-2 Delete Chapter 8 of book CLOB using locators

Definitions:

LOB-LOCATOR-1 USAGE IS SQL TYPE IS CLOB-LOCATORLOB-LOCATOR-2 USAGE IS SQL TYPE IS CLOB-LOCATOR

START-POSITION PIC S9(9) USAGE IS BINARYEND-POSITION PIC S9(9) USAGE IS BINARY

Pseudo-Code:

EXEC SQLSELECT LOBINTO :LOB-LOCATOR-1FROM BASE_TABLEWHERE KEYCOL1 = :HV-KEYCOL1

END-EXEC

EXEC SQLSET :START-POSITION = POSSTR (:LOB-LOCATOR-1, ‘Chapter 8’)

END-EXEC

EXEC SQLSET :END-POSITION = POSSTR (:LOB-LOCATOR-1, ‘Chapter 9’)


END-EXEC

EXEC SQLSET :LOB-LOCATOR-2 = SUBSTR (:LOB-LOCATOR-1, 1, :START-POSITION - 1) CONCAT

SUBSTR (:LOB-LOCATOR-1, :END-POSITION)END-EXEC

EXEC SQLUPDATE BASE_TABLESET LOB = :LOB-LOCATOR-2WHERE KEYCOL1 = :HV-KEYCOL1

END-EXEC

EXEC SQLFREE LOCATOR :LOB-LOCATOR-1, :LOB-LOCATOR-2

END-EXEC

A sample program showing how to delete a specific part of a LOB using locators is included in the SAMPLE6.TXT file described in Appendix D, “Additional material” on page 155.

Case 2: Updating a specific part of a LOBWhen you consider updating your LOB values in your application, you can build the new content of the LOB in the same way we have shown for insert cases in 3.4.2, “Inserting LOBs via host application” on page 34. The other way to apply necessary updates consists of locator usage. Using locators, you can simply replace parts of your LOB value or even delete them. Depending on the size of your LOBs and the number of updates you want to perform on a single LOB value, you have to decide when you want to use locators and when to replace a whole LOB value by building it in the application’s memory.

You can easily compare updating a part of a LOB with deleting a part of a LOB as shown in case one, except that the new locator is set up in a different way.

After determining the start and end position of your update, the only difference to Example 4-2 is the changed assignment of LOCATOR-2.

Example 4-3 Updating a part of a CLOB


:NEW-TEXT CONCATSUBSTR (:LOB-LOCATOR-1, :END-POSITION)

END-EXEC

So LOCATOR-2 is made of the former text referenced by LOCATOR-1 up to the start position minus one byte, the NEW-TEXT variable which can consist either of a host variable or a LOB locator, and the remaining text referenced by LOCATOR-1 from your end position up to the end of the LOB value

Using the technique mentioned above, you are also able to insert a certain text at a particular position in your CLOB. The only thing you have to figure out is the position where you want to insert the text in your CLOB.

A sample program showing how to update a specific part of a LOB using locators is included in the SAMPLE7.TXT file described in Appendix D, “Additional material” on page 155.


Let us now assume that you want to insert more text at the end of ‘Chapter 8’. For this reason you only have to find a position in the LOB where you want to place the new text value. After you have determined the correct position by using POSSTR, you can assign a new locator to the complete new value of the LOB. Example 4-4 shows you how you can perform an insert of new text to a known position in a LOB.

Example 4-4 Inserting a text at a particular position


:NEW-TEXT CONCATSUBSTR (:LOB-LOCATOR-1, :START-POSITION)

END-EXEC

In this example the NEW-TEXT variable can also be a host variable or another LOB locator.

4.3.2 Unloading a LOB via applicationUnloading a LOB value into a certain data set is easy if compared to the manipulating techniques described in 4.3.1, “Manipulating a LOB without retrieving it” on page 52. In general, the method you use to unload LOB values depends on the maximum available size of a host variable. If you are able to allocate a host variable being large enough to contain the maximum size of a LOB value you want to unload, unloading is easier than using locators.

Case 1: Unloading a LOB using one host variableAssume you want to unload a 10 MB CLOB value of a text document. The first thing you have to make sure is the format in which the data is needed, because you cannot write it to a single row in a data set. So the application has to split up the data into the requested file format. In our example we assume an output file of LRECL 1024 using record format FB. So the application selects the entire LOB value into a host variable and writes the value of the host variable in 1,024 byte pieces to an output data set. Example 4-5 provides a pseudo-code to perform a LOB unload via application using one host variable.

Example 4-5 Unloading LOB data using one host variable

Definitions:

HV-LOB USAGE IS SQL TYPE IS CLOB (10M)

Pseudo-Code:

EXEC SQLSELECT LOBINTO :HV-LOBFROM BASE_TABLEWHERE KEYCOL1 = :HV-KEYCOL1

END-EXEC

PERFORM WRITE-DATA

:WRITE-DATAMOVE 1 TO BYTE-COUNTER

PERFORM UNTIL BYTE-COUNTER > HV-LOB-LENGTHMOVE HV-LOB-DATA (BYTE-COUNTER:OUTPUT-FILE-LENGTH) TO OUTPUT-RECORDWRITE OUTPUT-RECORDADD OUTPUT-FILE-LENGTH TO BYTE-COUNTER


END-PERFORM

This solution works fine as long as the maximum LOB value is not bigger than the largest size of a host variable you are allowed to acquire. If you are not able to acquire a host variable of the size of your maximum LOB value, you can use locators for unloading your data as ontinued in Case 2.

A sample program showing how to unload a LOB using a host variable is included in the SAMPLE4.TXT file described in Appendix D, “Additional material” on page 155.

Case 2: Unloading an entire LOB using locatorsWhen you are not able to define a host variable of a size of the LOB data you want to unload, you can use a different technique. A method to unload huge amounts of data is using locators to reference a whole LOB value. The application only retrieves a part of a LOB which can be easily written to an output file. After processing the retrieved part, the next part is retrieved and also written to an output file. We assume the same file attributes as in Example 4-5. Example 4-6 gives you an idea how to unload a LOB value using locators when you are only allowed to use a 1 MB host variable and your LOBs are larger than 1 MB.

Example 4-6 Unloading a LOB using locators

Definitions:

HV-LOB USAGE IS SQL TYPE IS CLOB (1M)LOB-LOCATOR-1 USAGE IS SQL TYPE IS CLOB-LOCATOR

Pseudo-Code:

EXEC SQLSELECT LENGTH(LOB), LOBINTO :CURRENT-LENGTH, :LOB-LOCATOR-1FROM BASE_TABLEWHERE KEYCOL1 = :HV-KEYCOL1

END-EXEC

COMPUTE AMOUNT-FULL-SUBSTR = CURRENT-LENGTH / MAX-VAR-SIZECOMPUTE AMOUNT-REMAIN-SUBSTR = CURRENT-LENGTH - (AMOUNT-FULL-SUBSTR * MAX-VAR-SIZE)

PERFORM VARYING SUBSTR-COUNTER FROM 1 BY 1 UNTIL SUBSTR-COUNTER > AMOUNT-FULL-SUBSTREXEC SQLSET :HV-LOB = SUBSTR(:LOB-LOCATOR-1, 1, MAX-VAR-SIZE)

END-EXEC

PERFORM WRITE-DATAEND-PERFORM

IF AMOUNT-REMAIN-SUBST > 0 THENEXEC SQLSET :HV-LOB = SUBSTR(:LOB-LOCATOR-1,1,:AMOUNT-REMAIN-SUBSTR)

END-EXEC

PERFORM WRITE-DATAEND-IF


END-EXEC


Before we start retrieving parts of the LOB, the application sets a locator to a LOB value in order not to allow changes to the LOB value during the unit of work. Otherwise the content of the LOB can change while the application processes the LOB data. In this case we acquire a locator to refer to a frozen LOB value for our unit of work. The idea behind retrieving values via a locator is the same idea we use when we insert LOB value: try using large host variables to reduce the number of your SQL calls.

A sample program showing how to unload a LOB using a host variable of 1 MB and locators is included in the SAMPLE5.TXT file described in Appendix D, “Additional material” on page 155.

The variable AMOUNT-FULL-SUBSTR tells your program how often it has to perform the SUBSTR to retrieve a part of the LOB value of the size of your largest host variable. For the remaining bytes of the LOB we calculate AMOUNT-REMAIN-SUBSTR to retrieve those bytes which are not covered by the previously issued statements. After retrieving the results of each SUBSTR function, the data is written to a file using the WRITE-DATA subroutine as mentioned in case one.

This example for retrieving your LOB data avoids materialization of the LOB, because DB2 knows where the parts you want to retrieve via the SUBSTR function are stored. Therefore, DB2 uses the LOB pageset structure to locate the data pages you want to retrieve. For a detailed description of the LOB pageset structure, see 4.4.1, “Storage of LOBs” on page 60.

Figure 4-2 shows you how a delete can affect your unit of work when you do not use a locator technique to freeze a LOB value from your application’s view, or isolation levels that can protect you from this situation. In our specific example we assume cursor stability (CS) as the isolation level with CURRENTDATA(NO) for lock avoidance and RELEASE(COMMIT).

Figure 4-2 Accessing a LOB without a locator reference using ISOLATION (CS)

As you can see, every SELECT SUBSTR acquires an S-LOB lock for the time it takes to retrieve the requested value. As soon as DB2 has delivered the value (depending on your current settings for DB2 locking), it releases the S-LOB lock. The LOB lock is taken again by DB2 as soon as the next SELECT SUBSTR statement is issued. When you use this method, a second transaction is able to delete the LOB which is currently processed by another transaction.

SELECT SUBSTR (LOB, 1, 1024) S-LOB LockWHERE KEYCOL = Vala

SELECT SUBSTR (LOB, 1025, 1024) S-LOB LockWHERE KEYCOL = Vala

SELECT SUBSTR (LOB, 2044, 1024) SQLCODE 100WHERE KEYCOL = Vala

T1 T2

S-LOB Lock DELETE LOB WHERE KEYCOL = Vala

COMMIT


To avoid this kind of situation, use a locator to freeze the object you currently access. When your application assigns a locator to a particular LOB value, the lock is held by DB2 until you explicitly free the locator or issue DB2 COMMIT as you can see in Figure 4-3.

Figure 4-3 Processing a LOB using a locator reference

Transaction two acquires an S-LOB lock which is compatible with the S-LOB lock already held by transaction one. After transaction two issues a COMMIT and its thread terminates, the LOB is only visible for transaction one. The pages are finally deallocated when transaction one releases its S-LOB lock for the accessed value.

Case 3: Unload parts of a LOB using locatorsIn some cases your application may want to retrieve only a known portion of a LOB value. In this case you do not have to retrieve the entire LOB value, you can retrieve just parts of it. You can retrieve parts with and without locators. If you want to have a LOB’s part in your host variable for further processing, you can use the SELECT in case one including the LOB function to retrieve only a part of the value. But if even the part you want to retrieve is too big for one single host variable, you can use the pseudo-code mentioned in Case 2 to retrieve only the data you need.

4.3.3 Finding the nth occurrence of a stringSince the POSSTR function provides you with the ability to find the first occurrence of a string, you probably want to find the second or third position of your search string. DB2 allows you to use POSSTR function to succeed anyway, but you have to combine it with SUBSTR, since both functions you are able to quickly find the position you need. Only from the application point of view it is a bit more difficult than finding the first position. Example 4-7 provides a possible solution for searching a LOB value for the position of a search string you really need.

Example 4-7 Finding a specific occurrence of a string

EXEC SQLSET :POS = POSSTR (:LOB-LOCATOR, :SEARCH-STRING)

END-EXEC

[determine if correct position is returned]

S-LOB Lock DELETE LOB WHERE KEYCOL = Vala

COMMIT

end of UOW / thread

T1SELECT LOB INTO :LOB-LOCATOR S-LOB LockWHERE KEYCOL = Vala

SELECT SUBSTR (:LOB-LOCATOR, 1, 1024)

SELECT SUBSTR (:LOB-LOCATOR, 1025, 1024)

SELECT SUBSTR (:LOB-LOCATOR, 2049, 1024) . . .end of UOW / thread

T2


IF WRONG-POSITION THENMOVE 0 TO FINAL-POS

START-POS

PERFORM UNTIL CORRECT-POSITIONADD POS TO FINAL-POSCOMPUTE POS-START = FINAL-POS + STRING-LENGTH

EXEC SQLSET :POS = POSSTR (SUBSTR (:LOB-LOCATOR, :POS-START), :SEARCH-STRING)

END-EXEC

[determine if correct position is returned]END-PERFORM

END-IF

The first POSSTR statement returns as usual the first search string position in the LOB value ‘hiding’ behind a LOB locator.

Cursors and LOB valuesWhen you plan to vary between host variables, where you fetch your LOB values into, using a cursor (such as FETCH into host variable or FETCH into LOB locator variable), make sure that the CURRENT RULES special register has the correct value. CURRENT RULES STD lets you switch between host variables you fetch the cursor into. For more information about CURRENT RULES, see 3.3.2, “Impact on cursors fetching LOB values” on page 33.

List prefetch for LOBsWith respect to the size of LOBs, the only prefetch for LOBs DB2 performs is for a single LOB value. If a LOB occupies more than one page, LOB Manager will prefetch up to a chunk of pages at a time for the LOB value. You have to look at prefetching for LOBs as a different dimension from prefetching normal rows to reduce I/O overhead as much as possible.

4.4 LOBs are different DB2 objectsDealing with LOBs means dealing with new objects in your DB2 subsystem, and that includes new ways to do so. In the following sections we talk about differences between standard DB2 tables and the new LOB tables and columns.

Loading a LOB columnBecause the maximum LRECL for a file in z/OS or OS/390 is 32,760 bytes, large objects bigger than 32 KB cannot be loaded using the LOAD utility. An input file is not designed to contain a input variable spanned over more than one single row. For LOB values less than or equal to 32,756 bytes (you have to add a four byte length column preceding the data in the file) the LOAD utility can be used. For inserting LOB values bigger than 32,756 bytes you have to use an application program. But DB2 allows you to use the LOAD utility if you are going to load LOB columns up to 32,756 bytes even if the LOB column itself is declared as 2 GB. For additional information regarding LOAD, see 3.4.1, “LOAD with LOB columns smaller than 32 KB” on page 34 and 6.2.1, “LOAD” on page 94.

Note: You always LOAD the data into the base table and DB2 will automatically store the LOB data in the associated auxiliary table.


New lock sizesWith LOBs, DB2 V6 also introduced a new way to store huge amounts of data that can span over many pages. Common locking techniques acquire locks on table spaces, partitions, tables, pages or rows. Considering a LOB table space, there is only one table space, containing one table which possibly holds several LOB values. DB2 does not acquire a lock on many LOB data pages, nor on the entire table space. Therefore, two new lock sizes are introduced to efficiently handle locking for large objects:

� Shared LOB lock (S-LOB lock)� Exclusive LOB lock (X-LOB lock)

These new types of locks are also acquired via internal resource lock manager (IRLM). LOB locks are not related to any pages at all. DB2 takes both lock types explicitly by a combination of the LOB table space, the associated ROWID, and the LOB version number. For a more detailed description on locking for large objects and lock sequences, see 4.5, “Locking” on page 62.

Update mechanisms for LOBsWhen you update a large object, DB2 uses an updating technique different from the standard objects in DB2. Updating a LOB for DB2 means de-allocation of used data pages, and allocating and inserting new data pages which contain the new LOB value. For lock-related information on this topic, see 4.5.5, “Locks with UPDATE” on page 67, and for rollback-related information, see “Shadow Copy Recovery” on page 67.

Large objects and indexesFor obvious reasons a LOB column in the auxiliary table is not indexable. The maximum length of an index column is 255 bytes and LOBs are even bigger. Therefore you cannot create an index on a LOB column.

CompressionDB2 does not allow you to specify COMPRESS YES for LOB table spaces. Most objects stored in a BLOB column are already compressed anyway, like JPEG pictures or ZIP folders. Regardless of compression for a LOB table space, you can specify COMPRESS YES for the table space containing the base table.

EDITPROCs, FIELDPROCs, and VALIDPROCsLOB values cannot be compared, except with the LIKE predicate, and since they are not stored along with other columns, they are not available to any database procedures such as EDITPROCs, FIELDPROCs or VALIDPROCs. Note that although they are not available to these types of procedures, they are available to any triggers that are defined on the base table.

Check constraintsA check constraint cannot reference a LOB column.Those values are not designed to be eligible to work with constraints, because they are too large to check when changing a row’s content in a table.


Data capture and data propagation (DpropR)Even if LOB values have been defined with LOG NO, DpropR is able to replicate those values. The capture program reads the LOB descriptor to determine if any data in the LOB column has changed or not, and places an indicator in the capture data table. When the apply program reads the indicator, it then copies the entire LOB value, not just the changed parts of the LOB value. The apply program always copies the most current version of a LOB column directly from the source table, in our case the auxiliary table. So it replicates only full LOBs, parts of a LOB are not replicated.

Replication is not supported for DB2 Extenders for Text, Audio, Video and Image or other extenders where additional control files associated with the extender’s LOB column data are maintained outside of the database.

For further details on DpropR replicating LOBs, refer to DB2 UDB Replication Guide and Reference Version 7, SC26-9920.

4.4.1 Storage of LOBsLOB table spaces have an entirely new format. Because a LOB entry in a LOB column can span pages, pages have to be chunked together. A chunk is referred to as 16 pages of contiguous space acquired in a LOB table space. Depending on your size of data, a certain number of chunks is allocated. A single LOB value can be stored using many chunks also as using page allocations of less than 16 contiguous pages. Figure 4-4 shows you a possible storage of a single LOB value. The LOB in our example is stored in a chunk containing pages 1 to 16, pages 21 to 28, another chunk beginning at page 49 and at last four single pages starting at page 83.

Figure 4-4 LOB value spanned over pages using chunks and non-chunks

1

. . .

23

4

16

. . .

11 01 10 11 00

11 01 10 11 00

11 01 10 11 00

100110101101011001011001 . . .

21

. . .

2223

24

28

. . .

11 01 10 11 00

11 01 10 11 00

11 01 10 11 00

100110101101011001011001 . . .

49

. . .

5051

52

64

. . .

11 01 10 11 00

11 01 10 11 00

11 01 10 11 00

100110101101011001011001 . . .

83 84

8586

11 01 10 11 00

11 01 10 11 00

11 01 10 11 00

100110101101011001011001 . . .


LOB table space organizationStoring LOBs causes the table spaces containing their values to be organized in a slightly different way in order to support pageset structures for columns spanning the maximum available page size.

As in every other table space, a LOB table space has only one header page. The header page contains DB2 internal control information which is needed when you access your LOB data. The new type of pages are LOB space map pages. They are structured like a multi level index (an index containing several levels of leaf pages and are basically pointers to chunks and pages. You can find at least one LOB map page for every version of every single LOB value in your LOB table space. When we talk about de-allocation of LOB pages, the space map pages have set the invalid flag for all pages containing the LOB information, which includes both LOB map pages and LOB data pages.

LOB map pages contain descriptive information about LOB values, the chunk information itself is stored in LOB map pages, which point to the associated data pages for a LOB value. After reading the information stored in the LOB map pages, DB2 knows where to find the information it has to retrieve to access a particular LOB value. See Figure 4-5 for a possible LOB map page information about a single value.

Figure 4-5 LOB data pages chunked together via space map and LOB map pages

You can think of two different allocation units that a LOB map page points to. First, it can point to a chunk of data pages, which is nothing more than 16 pages of contiguous space (indicated by #P = 16 in Figure 4-4). As soon as DB2 uses partial chunks to store parts of the LOB value, it contains the page number where the allocation starts and the number of contiguous pages used.

503504

505

LOB Data Page

AllocationChunk/Page Page #P

Chunk 104 16

Chunk 410 16

Page 503 9

Page 602 8

Page 664 3

Space Map Page

LOB MAP Pages

104105

106

LOB Data Page

9

16

LOB MAP Page

. . .

. . .

}}


For a detailed description of LOB system pages, refer to the licensed documentation DB2 UDB for OS/390 and z/OS Version 7 Diagnosis Guide and Reference, LY37-3740.

4.5 LockingLocking considerations for LOBs are different from normal locking techniques for other DB2 objects. Because a single LOB can grow up to 2 GB and therefore it’s data is spanned over many pages, a new mechanism was introduced to ensure row consistency and data integrity while users are accessing LOB values. This is done using a combination of base table locks and the new LOB-locks, which were introduced in DB2 V6.

For the following examples and explanations we assume LOCKSIZE ANY. If you specify other lock sizes in your environment, the locked objects may differ.

The LOB lockA LOB lock is a new type of lock which is used to provide concurrent access on LOBs. These new types of locks support access to large objects for reading and updating applications without interfering with each other. You can find a shared LOB lock (S-LOB lock) and an exclusive LOB lock (X-LOB lock) as new lock types for large objects in your system. There is no need for an update LOB lock, because updating a LOB value always means deleting and inserting a value, so that there will not be any direct updates on LOB data pages. A LOB lock always locks an individual LOB and it comes with a corresponding lock on the base table to ensure row consistency. Locking both tables at access time belongs to the storage-concepts how data of one logical row is stored in two different tables. A LOB lock only occurs as a column lock on a single value.

There is no locking for untouched LOBs, because a data page cannot contain values of more than one single LOB value. But if you look at lock escalation mechanisms or if you explicitly specify LOCKSIZE TABLESPACE, you can find more locks than the number accessed by your application. Lock on LOB values are only escalated to the table space level, because there is only one table in each LOB table space which can be ignored for locking purposes. See Figure 4-6.

Note: When using DSN1PRNT, you may also see pages referring to LOB values that have been already deleted and whose space was not reused up to now.


Figure 4-6 Lock escalation on LOBS

4.5.1 Locks with simple readsIn this case LOBs differ from the objects in DB2. There is also a difference on how a LOB value is accessed, because the data describing the LOB is stored in a base table, while the LOB value itself is stored in an auxiliary table. When you select a LOB value into your host variable, you have to code your SQL statement on the base table. Accessing the auxiliary table via SQL is not supported. The first thing DB2 does is access the base table to find out which row in the auxiliary table it has to access.

For this access to the base table, an intent-share (IS) lock is needed on the table space containing the base table, on the base table itself and a share (S) lock on the page accessed while retrieving or validating the rows of the base table. After the particular LOB is located, a IS-lock on the LOB table space and a single S-LOB lock is acquired on the LOB value to ensure row consistency. The lock is held by DB2 until the whole value is delivered to your host variable, so that no updating transactions can interfere with your application and the LOB will not disappear while your application is reading it. Figure 4-7 gives you an idea of how DB2 serializes locks when accessing a single LOB value.

The numbers indicated in the figures in this section reflect the sequence in which the locks are taken.

Using lock avoidance or uncommitted read does not lock the accessed row in the base table while you are retrieving the LOB value. In this case, the part of a ‘logical row’ (containing the entry in the base table and the associated LOB value) which resides in the base table can be updated by concurrent transactions while you retrieve the associated LOB value.

Auxiliary Table

escalation

LOB table space

LOB table space lock

LOB lock


Figure 4-7 Scan lock sequence

4.5.2 Using ISOLATION (UR) for LOBsYou can use ISOLATION (UR) for your application, but DB2 has to request an S-LOB lock to ensure data integrity while the application retrieves the data. But what is DB2 doing when I access my LOB data even with ISOLATION (UR)? First, when using uncommitted read, an S-lock is acquired on the base table to prevent you from being hit by mass delete statements. This type of lock is also referenced as a mass delete lock. The next lock to be requested is an intent-share (IS) lock on the LOB table space and an S-LOB lock on the LOB itself. An S-LOB lock has to be requested to protect the LOB value from updates or deletes while the application retrieves the value. As mentioned before, a LOB value can span pages and it takes time to retrieve it; during this time no other transaction is allowed to change the value. Using S-LOB locks, even while using uncommitted read, prevents LOB data from being corrupted by someone else during the delivery process to your application by the LOB Manager. Figure 4-8 shows LOB lock serialization when you use uncommitted read.

Figure 4-8 Scan lock sequence using uncommitted read

Note: This differs from the general DB2 lock mechanism, where no parts of a row can be updated while it is delivered to your application. Using lock avoidance or uncommitted read delivers the row as it was at the point in time when your SQL statement was issued.

BASE Table Space: IS (1)

Page:S (3)

BASE Table: IS (2)

LOB Table Space: IS (4)

LOB:S (5)

BASE Table Space:

BASE Table: S Mass-Delete (1)


LOB:S (3)


4.5.3 Locks with INSERTInserting a LOB from a locking point of view requires more locks than inserting data in a normal table, because there are two tables involved every time during an insert process. At first, an intent exclusive (IX) lock is taken on the base table space and the base table. The same type of lock is acquired for the LOB table space after establishing the first two locks. After the three IX locks are established, an exclusive (X) lock is taken on the data pages which are used for inserting the LOB value. The last lock is an X-lock on the page in the base table where the new row is stored. The reason why insert applications get an X-LOB lock is to prevent access by scanners before the LOB is completely inserted. The actions performed by DB2 when you insert a LOB value from the locking point of view are shown in Figure 4-9.

Figure 4-9 Insert lock sequence

4.5.4 Locks with DELETEIn this section we differentiate between the locking operations occurring with a singleton delete and a mass delete.

Deleting a single LOB valueA LOB delete requires more locks than just those on the LOB table space, as you can see in Figure 4-10.

BASE Table Space: IX (1)

Page:X (5)

BASE Table: IX (2)

LOB Table Space: IX (3)

LOB:X (4)


Figure 4-10 Delete lock sequence

After establishing the usual IX locks on the base table space and the base table, only a intent-share (IS) lock is requested for the LOB table space. The page containing the associated data in the base table is also provided with an X-lock. There is no IX lock request for the LOB table space, because deleting a LOB value internally means de-allocating the allocated pages used by the LOB, and not directly updating the data pages. For this reason, the requested lock on the LOB is only a shared LOB (S-LOB) lock. If an S-LOB lock exists for a LOB value, the de-allocation is done after all other locks acquired by readers (see 4.5.1, “Locks with simple reads” on page 63 for locks requested by LOB readers) are released. So deleting applications acquire S-LOB-locks to reserve space in case of a rollback.

Even if the deleting application commits the unit of work or its thread ends, the COMMIT is done and the LOB is only accessible for the thread currently reading the LOB value.

Comparison with a mass deleteA mass delete also places the common IX-lock as seen in the examples above. For the base table, DB2 requires an X-lock and also a mass-delete X-lock on the base table. An X-lock for the LOB table space is acquired, too. See Figure 4-11 for a brief overview of locks taken at mass-delete time.

Figure 4-11 Locks acquired by mass delete


Page:X (3)

BASE Table: IX (2)


LOB:S (5)


BASE Table: IX (2) X (3) X mass delete (4)

LOB Table Space: X (5)


4.5.5 Locks with UPDATEDiscussing what happens when a LOB value is updated by an application program is quite easy after dealing with locks established at delete and insert times. A LOB update consists of a single LOB delete and another LOB insert, you can find both techniques only serialized when updating a LOB column. After DB2 takes both IX-locks on the base table space and the base table, it also requests an X-lock for a particular row in the base table. After all locks are done by the internal resource lock manager (IRLM), DB2 acquires an IS-lock on the LOB table space and an S-LOB lock to delete the old LOB. When the LOB is deleted, the new row is inserted after changing the IS-lock on the LOB table space to an IX-lock and an X-LOB lock is set for the new LOB value. You can find the sequence visualized in Figure 4-12.

Figure 4-12 Lock sequence when updating a LOB column

4.5.6 Some general information about lockingIn this section we add some considerations on LOB locking.

Conflicts between SELECTers and DELETErsAfter you have read the previous sections, you can ask about concurrency of selecters and updaters. While readers and deleters both take S-LOB locks, they can coexist without interfering with each other. The pages where the LOB data resides are only de-allocated if there are no more S-LOB locks on the LOB, which simply means that nobody accesses the LOB at this particular point in time.

Shadow Copy RecoveryWhen we talk about a shadow copy recovery, we talk about de-allocation and re-allocation of data pages in a LOB table space. When you delete a LOB value, the deletion de-allocates pages and logs only the de-allocation. The LOB data is left in the pages within the LOB table space as a shadow copy of the deleted data. A rollback simply results in re-allocation of the previously deleted pages. The S-LOB lock taken at delete prevents a later insertion from re-using the space de-allocated by the deletion. The S-LOB lock does not prevent deletions of that LOB but ensures that the space allocated to the LOB is not reused until the LOB lock is released. If the LOB is deleted, the shadow copy of that version of the LOB will persist until all LOB locks on that LOB version are released.


Page:X (3)

BASE Table: IX (2)

LOB Table Space: IS (4), IX (6)

LOB:S (5)

LOB:X (7)

Old New


When no LOB locks are takenThere are some situations where DB2 knows that it does not make sense to ask IRLM for a lock. There are at least four reasons in DB2 V7 when no LOB locks are acquired.

� Selecting a LOB value that is NULL or zero length

� Deleting a LOB value that is NULL or zero length

� Inserting a LOB value that is NULL or zero length

� Updating a LOB value that is NULL or zero length to zero length or NULL

Changes on base table rows are logged as usual, only indicator column changes are logged with the base table log record.

4.6 Details about ROWIDThere are two different ways of defining a column to be a ROWID data type in a CREATE TABLE statement:

� COLNAME ROWID GENERATED ALWAYS� COLNAME ROWID GENERATED BY DEFAULT

Using the GENERATED ALWAYS keyword, DB2 always generates a ROWID when inserting a row. Applications and users are not allowed to insert a ROWID.

If you use GENERATED BY DEFAULT, users and applications can supply a value for a ROWID column as long as the value was previously generated by DB2 and a unique, single column index that exists on the ROWID column. DB2 checks that the value you are going to insert is a valid ROWID. It is not sufficient to provide unique numbers yourself. You should only use this parameter when inserting data from another table for purposes of moving data. The recommended usage is GENERATED ALWAYS.

As mentioned above, you have to create a unique index on the ROWID column when you specify GENERATED BY DEFAULT.

Make sure that there is no way to use the GENERATED ALWAYS clause before implementing GENERATED BY DEFAULT, because the additional index on a table may increase your response time for inserting and deleting transactions on the base table. The index is not affected by an UPDATE statement since the ROWID is not updateable. If you try to update a ROWID column, DB2 issues SQLCODE -151, because the catalog description indicates that this column cannot be updated.

Be aware that a ROWID column implies some restrictions, preventing the values in the column from being manipulated:

� Users are not allowed to update a ROWID column.

� Null values can not be assigned to ROWID columns.

� EDITPROCs, FIELDPROCs and CHECK CONSTRAINTs are not provided for ROWIDs.

� It is not allowed to load a single partition or a range of partitions if a column of data type ROWID is part of the partitioning key.

Attention: When you specify GENERATED BY DEFAULT for a ROWID column, make sure that a single column unique index exists on your ROWID column. ROWID values can never contain null values, so the ROWID column has to be defined as NOT NULL.


The ROWID column is stored like a VARCHAR (17) column. In DB2 V7 two different types of ROWIDs can be defined.

To give you a better understanding of the different occurrences of a ROWID consider the following scenarios:

Case 1: A new table is created including a ROWID columnFirst, we have a look at a table where the CREATE TABLE statement already contains a ROWID column (for all examples, it does not matter if the ROWID column is defined using GENERATED ALWAYS or GENERATED BY DEFAULT) as shown in Example 4-8.

Example 4-8 DDL for a table containing a ROWID column

CREATE TABLE CUSTOMER ( CUSTNO CHAR(10) NOT NULL WITH DEFAULT

, CUSTNAME CHAR(32) NOT NULL WITH DEFAULT , ROW_ID ROWID NOT NULL GENERATED ALWAYS );

Now you insert a row into the CUSTOMER table, as shown in Example 4-9, without providing the ROWID column, because it will be generated by DB2 at insert time.

Example 4-9 Inserting a row in CUSTOMER table

INSERT INTO CUSTOMER (CUSTNO, CUSTNAME)VALUES (‘0000000018’, ‘ANNIKA THOMAS’);

After a new table is created, all CUSTNOs being inserted are associated with a unique ROWID value. After you insert a row (at this point in time a ROWID is associated with the inserted row), a SELECT on ROWID returns the following result set, as shown in Example 4-10.

Example 4-10 ROWID value of a table created with ROWID column

CUSTNO CUSTNAME ROW_ID ---------+---------+---------+---------+---------+---------+---------+---------+0000000018 ANNIKA THOMAS 63C6AB6415CED248260401D370140100000000000201---------+---------+---------+---------+---------+---------+---------+---------+

As you can see, the generated ROWID is externalized as a 44-byte value, but stored as VARCHAR (17).

When you use the DSN1PRNT utility, you can see the following HEX values for the above mentioned ROWID shown in Example 4-11:

Example 4-11 DSN1PRNT of ROWID in hex value

000E 63C6AB64 15CED248 260401D3 7014

The value ‘000E’ declares the length of the ROWID column, which is currently 000E in hex and 14 in decimal. Comparing to the information stored in the DB2 catalog that ROWIDs are stored as VARCHAR (17) columns, there are three bytes left for future extensions of ROWIDs.

Case 2: Adding a ROWID column via ALTER TABLEThings look different if a ROWID column is added when a table already contains many rows. Consider the table mentioned in Example 4-8 without a ROWID column and the table already contains lots of rows. Let us issue an ALTER TABLE statement as shown in Example 4-12.


Example 4-12 ALTER TABLE adding a ROWID column

ALTER TABLE EXAMPLE ADD ROW_ID ROWID NOT NULL GENERATED ALWAYS;

The ALTER TABLE statement does not affect any columns stored in the table, so no ROWID is stored up to now. If you do a SELECT on ROWID, you will receive a result set like the one in Example 4-13.

Example 4-13 ROWID value of a table where a ROWID column was added

CUSTNO CUSTNAME ROW_ID---------+---------+---------+---------+---------+---------+---------+0000000018 ANNIKA THOMAS 40000000800001300022020100000000000201---------+---------+---------+---------+---------+---------+---------+

As you can see, the ROWID now only consists of 38 bytes, compared to the previously mentioned ROWID of 44 bytes.

If you use DSN1PRNT again to look into your table space, you will not find any value for the ROWID column inside your table space. When you select the ROWID value, it is only generated at SELECT time, and stored inside your host variable. You can select the ROWID for a specific row several times, and the value in the ROWID column never changes.

The first time you update the row, the ROWID is physically stored in the table, with the same value delivered to you by DB2 when you selected the row before.

When you use DSN1PRNT again after you performed an UPDATE on the row, you will see results similar to Example 4-14.

Example 4-14 DSN1PRNT of ROWID in hex value after adding and updating a row

000B 40000000 80000130 002202

Now the value ‘000B’ declares the length of the ROWID column, which means 11 in a decimal value. Adding the two bytes length field, as usual, we now have a ROWID column made of 13 bytes.

So if an already existing row is updated and no ROWID value is stored for the updated row up to now, the new ROWID is at least 11 bytes plus the two bytes length field.

Let us now have a close look at the new rows which are inserted after the ROWID column was added to the table. All rows inserted after the ROWID column was added to the table have a ROWID of length X’000E’. This means 14 bytes plus two additional bytes for the length field.

The first row shown in Example 4-15 represents the ROWID value for an ‘old’ row being updated after ROWID was added to the table. It is important to know that this row already had been in the table before the ROWID column was added. The second row shows the ROWID value for a new row inserted after the ROWID column was added.

Example 4-15 ROWID values of updated and inserted columns

CUSTNO CUSTNAME ROW_ID ---------+---------+---------+---------+---------+---------+---------+---------+

Note: A ROWID column must be defined using NOT NULL. When you add a ROWID column, the NOT NULL attribute contradicts the normal usage of ALTER. In fact when ALTERing non-ROWID columns, you must specify NOT NULL WITH DEFAULT.


0000000018 MRS. THOMAS 40000000800001300022020100000000000201 0000000026 MR. KLINGEN 3F0DA87C95CED612260401D370180100000000000202---------+---------+---------+---------+---------+---------+---------+---------+

In Example 4-16 you can find the DSN1PRNT output for both ROWID values:

Example 4-16 DSN1PRNT of updated and inserted ROWIDs in hex value

000B 40000000 80000130 002202000E 3F0DA87C 95CED612 260401D3 7018

As indicated above, the pre-existing row has a ROWID length of ‘000B’ which is 11 bytes, the newly inserted row uses 14 bytes (‘000E’ as length field) to store the ROWID.

In conclusion, it is possible that no ROWID value is stored in the table space even if a ROWID column exists in the table, and you can find two different occurrences of ROWIDs even if they are stored in the table, depending on when the ROWID column was added to the table and whether or not the row existed at that time.

ROWID is a data type that can be used outside LOBs. ROWID gives you guaranteed unique values. This can be useful for applications and table designs where artificial keys have to be generated by the application to ensure uniqueness of a particular row. Using a ROWID column, DB2 is able to handle this special requirement for you. The ROWID behavior, that the first bytes in general appear to be pseudo-random, makes a ROWID column a good choice for a partitioning key if you want to spread your data randomly across partitions.

Note: Using a ROWID as a partitioning key is not a good idea when you have to process your data in the order of another key value!


Chapter 5. DB2 Extenders and LOBs

In this chapter we provide a brief description of DB2 Extenders available for the z/OS environment. We then look at the set up and administration of these extenders with regard to LOB utilization. The chapter is structured as follows:

� Generalities� AVI Extenders� XML Extenders� XML and LOBs

5


5.1 GeneralitiesDB2 for OS/390 has introduced extenders to enhance the usability and functionality of the RDMS.

They can be divided into text and AVI extenders for text, audio, video, image, and XML extenders for XML.

These extenders define new data types and functions using DB2 UDB for OS/390’s built-in support for user defined functions (UDF) and user distinct types (UDT).

They handle new non traditional data types in advanced applications, improving application development and reducing complexity.

Figure 5-1 shows the general DB2 Extender architecture.

Figure 5-1 Extender architecture

5.2 AVI ExtendersThe DB2 Audio, Video, and Image Extenders enable you to store image, audio, and video files in database tables alongside other traditional character and numeric data that are relevant to your data model. These extenders provide additional data types specifically for large image audio, and video file types and specialized query functions for altering, analyzing, and retrieving them.

Architectural view of DB2 Extenders

DB2 Client/CAE

DB2 Client Function

SQLSQL

DB2 UDB ServerDB2 UDB Server

ExternallyStored

MM Data

Servers

InternallyStored

MM Data

BLOB

Search

SupportData

MM

AttributesData

BusinessData

Business

Data

UDT

Stored

Proc.

UDF

ApplicationsApplications

BusinessBusinessDataData

StaticStatic Data Data

StaticStaticDataData

Stream edStreamedDataData

Extended SQL APIExtended SQL APICAPICAPI


Features of the Image Extender� Import and export images and their attributes into and out of a database. When you import

an image, the DB2 Image Extender stores and maintains image attributes such as size in bytes, format, height, width, and number of colors.

� Controls access to images with the same level of protection as traditional business data.

� Convert the format of images. You have the flexibility of importing or exporting an image in its source format or converting the format of the image when importing or exporting. You can also scale an image, rotate it, do black-white image inversion, or change representational characteristics, such as bits per sample and compression type.

� Secure and recover images. Images and their attributes that you store in a DB2 UDB database have the same security and recovery protection as traditional business data.

� Query images based on related business data or by image attributes. You can search for images based on data that you maintain, such as a name, number, or description; or by data that the DB2 Image Extender maintains, such as the format of the image or its distribution of colors. Now you can create a QBIC query easier than ever before.

� Generate and display image thumbnails and full images. A thumbnail is a miniature version of an image. When you import an image into a database, the DB2 Image Extender creates and stores a thumbnail of the image. You can use the DB2 Image Extender to retrieve a thumbnail or a full-size image. You can then use the DB2 Image Extender to invoke your favorite browser to display the thumbnail or full-size image.

� The DB2 Image Extender supports a wide variety of image formats, such as GIF, JPEG, BMP, and TIFF.

Features of the Audio Extender� Import and export audio clips and their attributes into and out of a database. When you

import an audio clip, the DB2 Audio Extender stores and maintains audio attributes such as number of audio channels, transfer time, and sampling rate.

� Secure and recover audio data. Audio clips and their attributes that you store in a DB2 database are afforded the same security and recovery protection as traditional business data.

� Query audio clips based on related business data or by audio attributes. You can search for audio clips based on data that you maintain, such as a name, number, or description; or by data that the DB2 Audio Extender maintains, such as the format of the audio or the date and time that it was last updated.

� Play audio clips. You can use the DB2 Audio Extender to retrieve an audio clip. You can then use the DB2 Audio Extender to invoke your favorite audio browser to play the audio clip.

� The DB2 Audio Extender supports a variety of audio file formats, such as WAVE and MIDI, and can work with different file-based audio servers.

Features of the Video Extender� Import and export video clips and their attributes into and out of a database. When you

import a video clip, the DB2 Video Extender stores and maintains video attributes such as frame rate, compression format, and number of video tracks.

� Secure and recover video data. Video clips and their attributes that you store in a DB2 database are afforded the same security and recovery protection as traditional business data.

� Query video clips based on related business data or by audio attributes. You can search for video clips based on data that you maintain, such as a name, number, or description;

Chapter 5. DB2 Extenders and LOBs 75

or by data that the DB2 Video Extender maintains, such as the format of the video or the date and time that it was last updated.

� Play video clips. You can use the DB2 Video Extender to retrieve a video clip. You can then use the DB2 Video Extender to invoke your favorite video browser to play the video clip.

� The DB2 Video Extender supports a variety of video file formats, and can work with different file-based video servers.

The installation of the audio, video, image extenders involves some effort in configuring the environment, both on the desktop, and the OS/390 platform.

5.2.1 DesktopThe desktop configuration includes the use of DB2 connect either Personal or Enterprise edition, and the installation of the AVI extenders. After the installation, the environmental variables must be set up through Win NT Settings-> Control Panel-> Environment Tab to define the path which contains the multimedia files.This path is used by the extender to retrieve and store multimedia files.

5.2.2 DatabaseOn the mainframe, the configuration includes building a database MMDBSYS for holding the administration tables for use by the extenders. Then binding the plans and packages used by the extenders.

After installing the extenders, you must bind the packages through DB2 connect issuing the following command from the CLP:

bind path/dmbmvsb3.lst grant public isolation cs

The table spaces used for storing the LOB data should be defined in the MMDBSYS database.

5.2.3 WLM environmentYou must add the steplib for the extender object modules to the WLM started tasks, along with creating a DMBENVAR DD card pointing to the USS path for the DB2 extenders created during the DB2 install. Example 5-1 shows the sample DMBWLM1 procedure.

Example 5-1 Sample DMBWLM1 procedure

//V71AWLM1 PROC DB2SSN=V71A,NUMTCB=18,APPLENV=DMBWLM1 //TCBNUM1 EXEC PGM=DSNX9WLM,TIME=1440, // PARM='&DB2SSN,&NUMTCB,&APPLENV',REGION=0M //STEPLIB DD DSN=DSN710.SDSNLOAD,DISP=SHR <== DB2 Load Library // DD DSN=DMB.V71.LOADLIB,DISP=SHR <== DB2EXT Load library //CEEDUMP DD SYSOUT=H //SYSPRINT DD SYSOUT=H //SYSDUMP DD SYSOUT=H //DMBENVAR DD DISP=SHR,DSN=&HQL.DMB.ENVAR <= DB2EXT ENV Variable File //*DSNAOTRC DD DISP=SHR,DSN=&HQL.DMB.DSNAOTRC <= DB2 CLI Trace

Important: PTF UQ63855 for APAR PQ54511 resolves a problem on ENABLE COLUMN encountered going through the DB2CLP Extender accessing from the workstation.


The following example is a sample of the contents of global environment variables defined in file DMB.ENVAR:

Example 5-2 DMBENVAR DD card input

DB2MMPATH=/usr/lpp/db2ext_07_01_00/samples:/tmpDB2MMTEMP=/tmpDB2MMSTORE=/tmpDB2MMEXPORT=/tmp

5.2.4 Administration tasks to enable a table to use extenders The following list is an ordered summary of the administration tasks you perform when you use the extenders the first time. You use DB2 commands or statements to perform some tasks. You perform other tasks with the DB2 extenders. This sequence assumes that your DB2 system is running:

1. Connect to the database server using the Extender CLP

2. Enable the database server using the Extender CLP

3. Create a table and column using DB2

4. Enable a table in the database using the Extender CLP

5. Enable a column in the table using the Extender CLP

ENABLE SERVER commandBy issuing the ENABLE SERVER command, through the DB2EXT.CLP, necessary UDTs are created for use by the extender functions, such as DB2AUDIO, DB2VIDEO, and DB2IMAGE.

An example of the ENABLE SERVER command is listed in Example 5-3.

Example 5-3 Example of ENABLE SERVER command

ENABLE SERVER FOR (DB2IMAGE,DB2AUDIO,DB2VIDEO) Using tablespace-name WLM ENVIRONMENT XXXXXXXX external security (USER,DB2)

Table definitions include these UDT's as the column characteristics. Table 5-1 shows the UDT’s used with the AVI extenders.

The DB2 Image and Audio Extenders define user-defined types (UDTs), or distinct types, and user-defined functions (UDFs) for storing and manipulating image and audio files. By using the DB2 support for large object types (LOBs), the extenders define the distinct data types DB2IMAGE, DB2AUDIO,and DB2VIDEO. For each of these types, the actual contents of the image, audio, or video file that they encapsulate can be stored as a binary large object (BLOB) in the database table, or outside of the database in a file system. In both cases, the contents of these distinct types are stored in an administrative support table that is maintained by the extenders. Only a character string called a handle that represents the type is actually stored in the user table. If the file is stored as a BLOB, it is stored in a column of an administrative support table. If the file is stored outside of the database, a file identifier (or pointer) to that file on the file system is stored in an administrative support table.


Table 5-1 UDTs used with AVI Extenders

In response to the above ENABLE SERVER command, the Image Extender:

� Creates a user-defined type that is named DB2IMAGE for image objects� Creates a user-defined type that is named DB2AUDIO for audio files� Creates a user-defined type that is name DB2VIDEO for video clips� Creates administrative support tables for image,audio, and video objects� Creates user-defined functions for multi media objects

The UDFs will run in the MVS Workload Manager (WLM) environment that is named DMBWLM1.

The tablespace name is the name of a table space used to store support tables and their indexes. It must be defined in the MMDBSYS database. The external security parameter is either:

� USER: This allows each UDF to execute as if the primary authid is the owner of the function.

Or,

� DB2: This uses the internal DB2 security and it is the default.

To enable tables to use the extender functions, the UDFs, UDTs, and TRIGGERS need to be created to administer the use of LOB columns in tables.

The extenders provide UDFs for storing these media data types. Through parameters to the supplied functions, you specify the content of the media file, its file format, and whether to store the file as a BLOB in the database or outside the database in the file system. The content of a file can be a buffer or file located on the client machine from where the function is called or a file on the file system where the database server resides.

User Defined Types created by DB2 AVI extenders

UDT Source Type Description

DB2IMAGE Varchar(250) Image handle — A variable length string that contains information needed to access an image object. Image handles are stored in a user table column enabled for the Image Extender.

DB2AUDIO Varchar(250) Audio handle — A variable length string that contains information needed to access an audio object. Audio handles are stored in a user table column enabled for the Audio Extender.

DB2VIDEO Varchar(250) Video handle — A variable length string that contains information needed to access a video object. Video handles are stored in a user table column enabled for the Video Extender.


Creating a table using the DB2IMAGE UDTExample 5-4 shows how to define a table using the DB2IMAGE UDT.

Example 5-4 Defining the table for an image

CREATE TABLE employee /*name of the table*/ (id CHAR(6) /*employee identification*/ name VARCHAR(40) /*employee name*/ picture DB2IMAGE) /*employee picture*/

ENABLE TABLE commandAfter the server has been enabled, and a table created using the appropriate UDT, the enable table command must be issued from the client DB2EXT.CLP to allow the table to store the media in a specific table space. The enable table statement is shown in Example 5-5.

Example 5-5 Example of ENABLE TABLE command

enable table table_name for extender_name using tablespace_name,,tablespace_name

In response to the ENABLE TABLE command, the Image Extender:

� Identifies the table for use.

� Creates administrative support tables that hold attribute information for image objects in enabled columns. The administrative support tables are stored in a table space that is named in the enable table statement and defined in the MMDBSYS Database. A 32 KB bufferpool is necessary for this table space.

� Creates an auxiliary table to hold LOB data for enabled columns. The auxiliary table is stored in a LOB table space that is named as the second table space in the enable table statement and is defined in the MMDBSYS Database.

� Creates indexes for the administrative support tables and the auxiliary LOB table.

ENABLE COLUMN commandAfter the server and table have been enabled to allow for the storage of the correct type of media, the table column is enabled to allow for media storage. The enable column statement is shown in Example 5-6.

Example 5-6 Example Enable Column Command

enable column table_name col_name for extender_name

In response to the ENABLE COLUMN command, the Extender:

� Identifies the column for use.

� Creates triggers.

These triggers update various administrative support tables in response to insert, update, and delete operations on the table.

Inserting LOB dataA program calling the appropriate UDF is used to insert data into either the LOB columns defined in the database, or to point to a file on the file system.


Example 5-7 shows the SQL statements in a C application program, which request an Image Extender UDF named DB2Image to store an image in a database table; the content of the source image is in a server file. It is accessed within the UDF by using a file reference variable. See A.3, “Using file reference variables” on page 144.

Example 5-7 Storing an image

EXEC SQL BEGIN DECLARE SECTION; long hvStorageType; EXEC SQL END DECLARE SECTION; hvStorageType=MMDB_STORAGE_TYPE_INTERNAL EXEC SQL INSERT INTO EMPLOYEE VALUES( '128557', /*id*/ 'Anita Jones', /*name*/ DB2IMAGE( /*Image Extender UDF*/ CURRENT SERVER, /*database server*/ '/employee/images/ajones.bmp', /*image content*/ file reference 'ASIS', /*keep the image format*/ :hvStorageType, /*store image in DB as BLOB* 'Anita''s picture') /*comment*/ );

When insert, update, and delete operations are performed on the tables defined within the extender, triggers update various administrative support tables in response. These support tables contain additional information about the extenders. Some of the tables identify user tables and columns that are enabled for the Extender. Other support tables contain attribute information about objects in enabled columns.

You use extender application programming interfaces to display images and play audio or video objects. You code these APIs using client function calls in C. The functions are run in the client.

Example 5-8 shows C statements which include an API that is named DBiBrowse. The API retrieves the data for an image handle and starts a browser to display the image.

Example 5-8 Browsing an image

EXEC SQL BEGIN DECLARE SECTION; char hvImg_hdl[251]; EXEC SQL END DECLARE SECTION EXEC SQL SELECT PICTURE INTO :hvImg_hdl WHERE NAME= 'Roger Rabbit'; rc=DBiBrowse( "ib %s", /*image browser*/ MMDB_PLAY_HANDLE, /*use image handle*/ hvImg_hdl, /*image handle*/ MMDB_PLAY_NO_WAIT); /*run browser independently*/

If the actual files are stored as BLOBs, they are provided the same security, concurrency, and integrity constraints as any other type of data inside the control of DB2. Users must have the required privilege to select, insert, or update the contents or the files, as well as the additional metadata that the extenders store about the image or audio data. These files and their metadata also can be backed up and recovered in the same way as any other data in DB2.

For more information on the AVI extenders go to:

http://www.ibm.com/software/data/db2/extenders/imgfmt.htm


5.3 XML ExtendersThe XML Extender is a new feature of DB2 and a successful implementation requires careful planning. The XML Extender provides the following user-defined types (UDT) for use with XML columns:

� XMLVarchar � XMLCLOB � XMLFILE

Table 5-2 shows the default UDTs and the length assignment after enabling the XML SERVER. If the CLOB length must be expanded, you need to drop each stored procedure, and recreate them with an increased CLOB limit.

Table 5-2 DEFAULT UDTs after enabling the XML SERVER

All the XML Extender facilities supplied for application programs run in the OS/390 MVS environment as stored procedures or user-defined functions (UDFs). User-defined types (UDT) are data types created by a DB2 application or tool. These data types are used to identify the storage type of XML documents in the application table. You can also store XML documents as files on a local file system specifying a file name.

All the XML Extender's user-defined types have the qualifier DB2XML, which is the schema name of the DB2 XML Extender user-defined types. For example:

db2xml.XMLVarchar

The XML Extender provides storage functions to be used with the XML Column storage type. Table 5-3 shows the storage functions and the base file type associated with each.

Table 5-3 XML Extender storage functions

The DB2 XML Extender provides powerful UDFs to store and retrieve XML documents in XML columns, as well as to extract XML element or attribute values. A UDF is a function that is defined to the database management system and can be referenced thereafter in SQL statements.

The XML Extender provides the following types of UDFs:

� Storage

It stores intact XML documents in XML-enabled columns at XML data types.

Schema Data TYPE NAME

SCHEME SOURCE DATA TYPE

LENGTH

DB2XML XMLCLOB SYSIBM CLOB 2147483647

DB2XML XMLVARCHAR SYSIBM VARCHAR 3000

Db2XML XMLFILE SYSIBM VARCHAR 512

The XML Extender storage functions

Basetype Stored in DB2 as:

XMLVCHAR XMLCLOB XMLFILE

Varchar XMLVCAHR N/A XMLFileFromVarchar()

CLOB N/A XMLCLOB XMLFileFromCLOB()

File XMLVarcharFromFile() XMLCLOBFromFile() XMLFILE


� Extract

It extracts XML documents, or the values specified for elements and attributes as base data types.

� Update

It updates entire XML documents or specified element and attribute values.

Some of the UDFs that refer to the XMLFILE data type, require access to an HFS system. The DB2 XML trace file, is also written to an HFS file. Table 5-4 identifies the XML Extender storage UDFs, and their functions.

Table 5-4 Storage UDFs used with XML Extenders

Table 5-5 identifies the XML Extender extract UDFs, their functions, and returned values.

Table 5-5 Extract UDFs used with XML Extenders

The XML Extender storage UDFs

Storage user defined functions

Return type Function

XMLVarcharFromFile() XMLVARCHAR Reads an XML document from a file on the server and returns the value of the XMLVARCHAR type.

XMLCLOBFromFile() XMLCLOB Reads an XML document from a file on the server and returns the value of the XMLCLOB type.

XMLFileFromVarchar() XMLFILE Reads an XML document from memory as VARCHAR, writes it to an external file, and returns the value of the XMLFILE type, which is the file name.

XMLFileFromCLOB() XMLFILE Reads an XML document from memory as CLOB or a CLOB locator, writes it to an external file, and returns the value of the XMLFILE type, which is the file name.

XML Extender extract UDFs

TABLE FUNCTION RETURNED COLUMN RETURNED TYPE

extractinteger returnedInteger INTEGER

extractSmallint returnedSmallint SMALLINT

extractDouble returnedDouble DOUBLE

extractReal returnedReal REAL

extractChar returnedChar CHAR

extractVarchar returnedVarchar VARCHAR

extractClob returnedClob CLOB


5.3.1 XML ToolkitThe XML Extender requires IBM XML Toolkit for OS/390 (FMID HXML120) 5655-D44, a no-charge product. Some functions also require UNIX Systems Services. XML Toolkit for OS/390 is available on tape, or via a download from the Internet, to your PC, then uploaded to the host. The following files constitute the package, although SMP/E knowledge is required especially with regards the SMP/E GIMZIP function:

� XML Toolkit 1.2.0 for z/OS and OS/390 README file in text format:

XMLSMPE.README.txt (21.4KB)

� XML Toolkit 1.2.0 for z/OS and OS/390 package in pax format:

XMLSMPE.pax.Z (41.4MB)

� XML Parser 1.2.0 for z/OS and OS/390, C++ Edition:

XMC331B.tar (16.5MB)

� XML Parser 1.2.0 for z/OS and OS/390, Java Edition:

XMJ311B.tar (3.8MB)

5.3.2 WLM application environmentYou need to set up a WLM Application Environment, running WLM in goal mode and build the procedure reported in Example 5-9. The key point is to include the XML Toolkit library in the STEPLIB.

Example 5-9 WLM procedure

//ISC3XML PROC RGN=0K,APPLENV=ISC3XML,DB2SSN=ISC3,NUMTCB=8 //IEFPROC EXEC PGM=DSNX9WLM,REGION=&RGN,TIME=NOLIMIT, // PARM='&DB2SSN,&NUMTCB,&APPLENV' //STEPLIB DD DISP=SHR,DSN=ISC710P1.RUNLIB.LOAD // DD DISP=SHR,DSN=SYS2.DB2.V710.SDSNLOAD // DD DISP=SHR,DSN=SYS2.DB2.V710.SDXXLOAD // DD DISP=SHR,DSN=SYS1.SIXMMOD1 <===================XML Toolkit Library//SYSIN DD UNIT=SYSDA,SPACE=(4000,(20,20),,,ROUND) //SYSPRINT DD UNIT=SYSDA,SPACE=(4000,(20,20),,,ROUND) //SYSTSPRT DD UNIT=SYSDA,SPACE=(4000,(20,20),,,ROUND)

5.3.3 General host requirementsDB2 V7 (HDB7710) and the XML Extender (JDB771X, Compid 5740XYR06) must both be installed in the same CSI.

RRS is required for XML Extender stored procedures and UDFs that run in WLM managed address spaces.

extractDate returnedDate DATE

extractTime returnedTime TIME

extractTimestamp returnedTimestamp TIMESTAMP

XML Extender extract UDFs


The Odb2 command line tool is required to allow SQL from the UNIX Shell. This can be downloaded and installed from the following Web site, and then select Tools and Toys.

http://www.s390.ibm.com/products/oe/

The PTF UQ52836 is highlighted as a requirement. It upgrades XML/390 beta code to GA. This is a must requirement.

5.3.4 Workstation requirementsInstall the XML Extender administration Wizard from:

http://www.ibm.com/software/data/db2/extenders/xmlext/downloads.html

The Administration Wizard also requires the following software on the client desktop:

� DB2 UDB Connect Personal or Enterprise Edition� JAVA Development Kit 1.1.7 or higher

The initial environment for XML Extenders is now complete for OS/390 or z/OS.

5.4 XML and LOBsDB2's XML Extender provides the ability to store and access XML documents, to generate XML documents from existing relational data, and to insert rows into relational tables from XML documents. XML Extender provides new data types, functions, and stored procedures to manage your XML data in DB2.

With the content of your structured XML documents in a DB2 database, you can combine structured XML information with traditional relational data. Based on the application, you can choose whether to store entire XML documents in DB2 as in user-defined types provided for XML data (XML data types), or you can map the XML content as base data types in relational tables. For XML data types, the XML Extender adds the power to search rich data types of XML element or attribute values, in addition to the structural text search that the DB2 Text Extender for OS/390 provides.

What XML Extender can do for your applicationsStore entire XML documents as column data or externally as a file, while extracting desired XML element or attribute values and storing it in side tables, which are indexed subtables used for high-speed searching. By understanding the elements and attributes most frequently searched, you can set up your side table structures. When planning for your sidetables, you must consider how many tables to create, how to organize them, and whether or not to create a default view for the tables. These decisions are partly based on whether or not you have multiple occurring elements or attributes. These sidetables are a subset of the existing tables, containing the same columns as those elements or attributes defined in the DAD.

Side tables are DB2 tables containing only a subset of the original tables, used to extract the content of an XML document that will be searched frequently. The location path of the element or attribute is mapped to a table and column, indexed, and used for searches. When the XML document is updated in the application table, the values in the side tables are automatically updated.

Figure 5-2 depicts the relationship between the XML document stored in the XML COLUMN and the side tables created to improve search performance.



http://www.s390.ibm.com/products/oe/

Figure 5-2 XML columns with side tables

For elements or attributes in an XML document that have multiple occurrences, you must create a separate side table for each XML element or attribute with multiple occurrences, due to the complex structure of XML documents. This means that elements or attributes that have multiple occurring location paths must be mapped to a table with only one column, the column for that location path. You cannot have any other columns in the table, whether or not they have multiple occurrences

Default views and query performanceWhen you enable an XML column, you can specify a default, read-only view that joins the application table with the side tables using a unique ID, called the ROOT ID. With the default view, you can search XML documents by querying the side tables.

The advantage of querying the default view is that it provides a virtual single view of the application table and side tables.

However, the more side tables are created, the more expensive the query. Therefore, creating the default view is only recommended when the total number of side-table columns is small. Applications can create their own views, joining the important side table columns.

By storing the documents as column data, you can:

� Perform fast search on XML elements or attributes that have been extracted and stored in side tables as SQL basic data types, and then create indexes on them.

� Update the content of an XML element or the value of an XML attribute

� Extract XML elements or attributes dynamically using SQL queries

� Validate XML documents during insertion and update


� Perform structural-text search with the Text Extender

How XML and DB2 work togetherXML Extender provides the following features to help you manage and exploit XML data with DB2:

� Administration tools to help you manage the integration of XML data in relational tables � Storage and access methods for XML data within the database

The XML Extender provides three UDTs: XMLVARCHAR, XMLCLOB, and XMLFILE. XMLVARCHAR and XMLCLOB stores the XML document as a VARCHAR and CLOB respectively, while XMLFILE stores the XML document as a file on a local file system.

The XML Extender allows you to store DTDs, the set of declarations for XML elements and attributes. When a database server is enabled for XML, a DTD repository table (DTD_REF) is created. Each row of this table represents a DTD with additional metadata information. Users can access this table to insert their own DTDs. The DTDs are used for validating the structure of XML documents.

You specify how structured XML documents are to be processed by the XML EXTENDER using a document access definition (DAD) file. The DAD file is an XML-formatted document that maps the XML document structure to a DB2 table. The DAD file specifies whether you are storing documents using the XML COLUMN method, or defining an XML COLLECTION for composition or decomposition.

These methods have very different uses, but can be used in the same application.

Storing whole XML documents as column data is known as the XML COLUMN method of storage and access. Composing or decomposing the contents of XML documents with one or more relational tables, uses the XML COLLECTION method of storage and access

XML columnThis method helps you store intact XML documents in DB2. The XML column method works well for archiving documents. The documents are inserted into columns that are enabled for XML and can be updated, retrieved, and searched. Element and attribute data can be mapped to DB2 tables (side tables), which can be indexed for fast search.

Figure 5-3 shows how DB2 can store a complete, tagged document in a CLOB column of a table using the XMLCLOB data type.


Figure 5-3 XML column

In Example 5-10 below, a small C program segment with embedded SQL (SQL statements coded within an application program) illustrates how an XML document is retrieved from a file to memory.

Example 5-10 Retrieval of an XML column document

This example assumes that the column ORDER is of the XMLFILE type.

EXEC SQL BEGIN DECLARE SECTION;SQL TYPE IS CLOB_LOCATOR xml_buff;EXEC SQL END DECLARE SECTION;EXEC SQL CONNECT TO SALES_DBEXEC SQL DECLARE c1 CURSOR FORSELECT Content(order) from sales_tabEXEC SQL OPEN c1;do {

EXEC SQL FETCH c1 INTO :xml_buff;if (SQLCODE != 0) {break;}else {/* do whatever you need to do with the XML doc in buffer */}

}EXEC SQL CLOSE c1;EXEC SQL CONNECT RESET;


XML collectionThis method allows for the storing of untagged XML data in one or more database tables. XML data is either decomposed from incoming XML documents or used to compose outgoing XML documents. If your data is to be shared with other applications, you may want to be able to compose and decompose incoming and outgoing XML documents and manage the data as necessary to take advantage of the relational capabilities of DB2. This type of XML document storage is called XML collection.

Figure 5-4 shows how DB2 stores data in the XML Collection which stores separate elements and attributes in different tables used in composing or decomposing documents.

.

Figure 5-4 XML collection

DTDThe Document Type Definition (DTD) lets us define the different pieces of data we plan to model, along with the relationships between them. The ability to include this semantic information is the source of XML's power, and its main advantage over HTML.

For an example of how to create a DTD, see:

http://www.ibm.com/developerworks/library/buildappl/writedtd.html#h2

DADOne of the inputs into both storage and retrieval is the user specified mapping file that creates the relationship between relational data and the XML document structure. This mapping file is called a document access definition (DAD) and provides a way to produce an XML document with the XML attributes and elements that you name as you please, and in any shape you want.


http://www.ibm.com/developerworks/library/buildappl/writedtd.html#h2

For an example of how to construct DADs and an explanation of their uses, see:

http://www.ibm.com/software/data/pubs/papers/db2webservices/db2webservices.pdf

UDTsThe XML Extender provides XML user defined types in which you define a column to hold XML documents. Table 5-6 defines these UDTs.

Table 5-6 User defined types

UDFsHere are a list of the UDFs in use by the XML extender for the manipulation of XML data within the RDMS.

� Storage

It stores intact XML documents in XML-enabled columns at XML data types.

� Extract

It extracts XML documents, or the values specified for elements and attributes as base data types.

� Retrieval

It retrieves data to either a CLOB stored in a database or to an external server file.

� Update

It updates the entire XML document or specified element and attribute values.

SecurityXML Documents stored as columns in DB2 tables are afforded the same security as any other user table column. When an administrator enables a database server for use with the XML Extender, he can specify which type of security to use. Either SECURITY USER, which assigns the primary ID of the process which calls the function to the XML UDF execution environment, or SECURITY DB2 which uses the ID of the WLM to access the XML UDF execution environment.

Use-defined type column Source data type Usage description

XMLVARCHAR VARCHAR(varchar_len) Stores an entire XML document as VARCHAR inside DB2. Used for small documents stored in DB2.

XMLCLOB CLOB(clob_len) Stores an entire XML document as CLOB inside DB2. Used for large documents stored in DB2.

XMLFILE VARCHAR(1024) Stores the file name of an XML document in DB2, and stores the XML document in a file local to the DB2 server. Used for documents stored outside DB2.



When a UDF attempts to access a file, USS calls an external security package to get the USERID associated with the UDF. For SECURITY USER, the file system checks against the USERID of the process that calls the UDF. Because you can assign users to groups and assign different authorities to different groups, you can control access to files on a group-by-group level.

SECURITY DB2 requires less administration in that the you need to assign an authorization ID, USERID, and Group ID to the WLM address spaces which run the UDFs. When you use the SECURITY USER you must assign a USERID and Group ID for every legitimate user of the files. In both cases you must coordinate the file system permissions with the Group and USERIDs.

Some XML Extender administrative functions require special authority.

Administrative functionsThe ENABLE or DISABLE SERVER command should be executed by the DB2XML userid which has SYSADM or DBADM.

When issuing the ENABLE or DISABLE COLUMN commands the administrator must have table owner privileges on the table which contains the column to be enabled. The administrator also has to have privileges for table spaces and buffer pools.

When using the ENABLE COLLECTION administrative command, the administrator must have table owner privileges on all tables in the collection and use of buffer pools and table spaces.

Composition and decomposition of documentsAn application developer needs access to the following tables:

� Composition

– DTD_REF (to access the DTD) – XML_USAGE (to access the DAD files) – SELECT on all tables referenced – INSERT on the result table

� Decomposition

– DTD_REF (to access the DTD) – XML_USAGE (to access the DAD files) – INSERT on all tables to be modified – UPDATE on all tables to be modified – SELECT on any table referenced in the DAD file – Access to the DB2 catalog

In conclusion, DB2 extenders provide the functionality to make LOBs useful within your application. By using the Extender APIs, it makes application development using these multimedia files easier and more structured.


Chapter 6. Data administration with LOBs

In this chapter we document the differences in administrating table spaces pertaining to tables containing LOB columns. We go through the various DB2 utilities and document their changes because of the introduction of LOB columns. The chapter is structured as follows:

� The DB2 Catalog and LOBs� Utilities� Recovery scenario� Operational scenario

6


6.1 The DB2 Catalog and LOBsIf you are using LOBs, the table space holding LOBs data is managed using statistics in SYSIBM.SYSTABLEPART, the same as the regular table spaces. In this section you will find a description of the additional columns that have been provided by DB2 V6 to the system catalog in order to support LOBs. This may help you navigating through the DB2 system catalog searching for LOB related information. We also briefly describe the LOB tables that are created within the DB2 system catalog by DB2 V7.

6.1.1 Some catalog definitions for LOBsIn this section we list the catalog tables and show the major DB2 catalog entries related to LOBs.

SYSIBM.SYSAUXRELSThe table SYSIBM.SYSAUXRELS shows you all relationships between your base tables and their related auxiliary tables created in your system to store LOB columns. When you have a partitioned base table space, the partition number is also indicated. The columns you can retrieve from the table are:

� TBOWNER, authorization ID of the owner of the base table� TBNAME, name of the base table� COLNAME, name of the LOB column in the base table� PARTITION, partition number when the base table space is partitioned, otherwise it is 0� AUXTBOWNER, authorization ID of the owner of the auxiliary table� AUXTBNAME, name of the auxiliary table� AUXRELOBID, internal identifier of the relationship between base table and auxiliary table� IBMREQD, internal DB2 release dependency indicator, for internal use only

SYSIBM.SYSCOLUMNSIf there are LOB columns defined within a base table, you will find them in the catalog table SYSIBM.SYSCOLUMNS as listed in Example 6-1.

Example 6-1 Select from SYSIBM.SYSCOLUMS

NAME TBNAME TBCREATOR COLTYPE LENGTH LENGTH2 -------- -------- --------- -------- ------ ----------- AUXVALUE SYSJARCLASS_SOURCE SYSIBM CLOB 4 10485760 AUXVALUE SYSJARDATA SYSIBM BLOB 4 104857600

� The TBNAME specified within the SYSIBM.SYSCOLUMNS row will indicate the name of the base table, because the LOB column is logically part of the base table.

� The field COLTYPE in the table will be set to indicate the data types of BLOB, CLOB, and DBCLOB.

� The LENGTH catalog column also will be set to a value of four for all LOB columns, because four bytes of internally defined information will be stored within each row of the base table for every defined LOB.

� The LENGTH2 column is set to the specified maximum length of the LOB column.


SYSIBM.SYSLOBSTATSThree fields from SYSIBM.SYSLOBSTATS can be used to manage space for LOB table spaces:

� FREESPACE, kilobytes of free space in extents with respect to high used RBA (HURBA)� AVGSIZE, average number of bytes per LOB� ORGRATIO, indication of optimal physical location of active LOBs

SYSIBM.SYSTABLEPARTThere are some more fields from SYSIBM.SYSTABLEPART, already used for regular table spaces, that can also be used to manage your LOB table spaces and its partitions:

� CARDF, the number of LOBs � SPACEF, KB of space allocated � PQTY, 4 KB primary extent allocation� SQTY, 4 KB secondary extent allocation� DSNUM, number of linear data sets for table space� EXTENTS, total primary and secondary extents

SYSIBM.SYSTABLESThe rows contained in SYSIBM.SYSTABLES which describe the auxiliary table and the base table are listed in Example 6-2.

Example 6-2 Select from SYSIBM.SYSTABLES

NAME CREATOR TYPE TSNAME RECLEN STATUS TABLESTATUS --------------- ------ ---- -------- ------ ------ ----------- BLOB_AUX_TABLE PAOLO X BLOBATS 0 BLOB_BASE_TABLE PAOLO T BLOBBTS 75

� RECLENGTH indicates the length of the stored record in the base table pages. It includes the actual four bytes of internally defined information that are stored within the base table for each defined LOB column. For the row describing an auxiliary table, the RECLENGTH field is set to 0.

� The value blank for the STATUS column indicates that the table has no parent index, and the definition is complete. The 'X’ instead indicates that the table has a parent index and that the table definition is complete. The value 'I’ indicates that the table definition is incomplete. The reason the table is incomplete is defined in the TABLESTATUS column.

� The TABLESTATUS contains 'L’ if the table definition is incomplete because an auxiliary table or auxiliary index has not been defined for a LOB column. It contains 'P’ if the table definition is incomplete because it lacks a parent index.

6.1.2 LOBs defined in DB2 catalogEven if you have no application using DB2 large objects in your environment, starting with V7, DB2 already defines LOBs in DBSNDB06. If you run a SELECT on SYSIBM.SYSAUXRELS using WHERE clause AUXTBOWNER = ‘SYSIBM’, you will see the two tables listed in Example 6-3. If you check their contents in the catalog, you will find out that tables SYSIBM.SYSJARCLASS_SOURCE and SYSIBM.SYSJARDATA contain LOB definitions.

Example 6-3 DB2 catalog query

SELECT COLNAME, PARTITION, AUXTBOWNER, AUXTBNAMEFROM SYSIBM.SYSAUXRELSWHERE AUXTBOWNER = ‘SYSIBM’

Chapter 6. Data administration with LOBs 93

COLNAME PARTITION AUXTBOWNER AUXTBNAME ------------------ --------- ---------- ------------------ CLASS_SOURCE 0 SYSIBM SYSJARCLASS_SOURCE JAR_DATA 0 SYSIBM SYSJARDATA

Both tables are related to the JAVA stored procedures implementation within DB2. SYSIBM.SYSJARDATA has a 100 MB BLOB column for the contents of JAR files and it is the auxiliary table for SYSIBM.SYSJAROBJECTS. The SYSIBM.SYSJARCLASS_SOURCE has a 10 MB CLOB for the contents of the class in JAR files and it is the auxiliary table for SYSIBM.SYSJARCONTENTS.

6.2 UtilitiesIn this section we discuss considerations related to LOBs when using the following utilities:

� LOAD� DSNTIAUL (this is a sample program, not a utility, but often used for unloading)� UNLOAD� COPY� REPORT TABLESPACESET� RUNSTATS� REORG� CHECK LOB� CHECK DATA� CHECK INDEX� DSN1COPY� RECOVERY

6.2.1 LOADBecause there is a limit of 32 KB for the size of the input record of the LOAD utility, you can use it only to LOAD LOB columns of 32 KB or less.

When the input record is greater than 32 KB, you have to load the LOB data separately. See the sample job DSN8DLPL in the SDSNSAMP DB2 distribution libraries for an example.

If you restart a LOAD job for a table with LOB columns that specified the RESUME YES option, you must use RESTART CURRENT.

If you use RESTART PHASE to restart a LOAD job which specified RESUME NO, the LOB table spaces and indexes on auxiliary tables will be reset.

For a table with LOB columns, you cannot restart a LOAD job that specifies the INCURSOR option.

Indexes on the auxiliary tables are not built during the BUILD phase. Instead, LOB values are inserted (not loaded) into auxiliary tables during the RELOAD phase as each row is loaded into the base table, and each index on the auxiliary table is updated as part of the INSERT operation. Because the LOAD utility inserts keys into an auxiliary index, free space within the index may be consumed and index page splits may occur. Consider reorganizing an index on the auxiliary table after LOAD completes to introduce free space into the index for future INSERTs and LOADs.


LOAD allows you to specify LOG YES or NO during their execution. A LOB table space can also be defined with LOG YES or LOG NO and this will affect logging while updating or loading a LOB column. Table 6-1 shows the effect on logging output and LOB table space with the possible combinations of the two parameters.

Table 6-1 Impact of logging on LOB tables

Loading LOB columns with input records that are < 32 KB works just like loading other non-LOB columns.

Example 6-4 shows you the output from a LOAD job, loading data into a table with a CLOB column defined. Notice that even though NOCOPYPEND was specified in the input cards, the load job places both the base and aux table in copy pending. Notice also that the base table had been defined in a compressed table space. This is allowed for the table space containing the base table, but not for the LOB table space. In this specific case the compression is not of much help because of the small number of rows and their small size. The LOB table space was defined with LOG NO.

Example 6-4 Output for Load job inserting into table with CLOB column

DSNU000I DSNUGUTC - OUTPUT START FOR UTILITY, UTILID = POLOR3LD DSNU050I DSNUGUTC - LOAD DATA LOG NO REPLACE INDDN SYSREC00 NOCOPYPEND DSNU650I -DB2G DSNURWI - INTO TABLE "PAOLOR3"."CLOB_BASE_TABLE" DSNU650I -DB2G DSNURWI - (DOCUMENT_NR POSITION(1) CHAR(10), DSNU650I -DB2G DSNURWI - DESCRIPTION POSITION(11) CHAR(32), DSNU650I -DB2G DSNURWI - DOCUMENT POSITION(43) CLOB) DSNU350I -DB2G DSNURRST - EXISTING RECORDS DELETED FROM TABLESPACE DSNU231I -DB2G DSNURBDC - DICTIONARY WITH 512 ENTRIES HAS BEEN SUCCESSFULLY BUILT FROM 21 ROWS FOR TABLESPACE PAOLOR3.CLOBBTS1 DSNU234I -DB2G DSNURWT - COMPRESSION REPORT FOR TABLE SPACE PAOLOR3.CLOBBTS1 1 KB WITHOUT COMPRESSION 1 KB WITH COMPRESSION 0 PERCENT OF THE BYTES SAVED FROM COMPRESSED DATA ROWS 0 PERCENT OF THE LOADED ROWS WERE COMPRESSED 72 BYTES FOR AVERAGE UNCOMPRESSED ROW LENGTH 72 BYTES FOR AVERAGE COMPRESSED ROW LENGTH 1 PAGES REQUIRED WITHOUT COMPRESSION 65 PAGES REQUIRED WITH COMPRESSION -6400 PERCENT OF THE DB2 DATA PAGES SAVED USING COMPRESSED DATA

Note: Apply PTF UQ65857 for APAR PQ59820. It solves a page addressing problem and improves LOAD performance.

Load and Reorg log impact on a LOB table

LOAD LOG keyword

LOB table space LOG attribute

What is logged LOB table space status after utility completes

LOG YES LOG YES Control information and LOB data

No pending status

LOG YES LOG NO Control information No pending status

LOG NO LOG YES Nothing COPY Pending

LOG NO LOG NO Nothing COPY Pending


DSNU304I -DB2G DSNURWT - (RE)LOAD PHASE STATISTICS - NUMBER OF RECORDS=21 FOR TABLE PAOLOR3.CLOB_BASE_TABLEDSNU302I DSNURILD - (RE)LOAD PHASE STATISTICS - NUMBER OF INPUT RECORDS PROCESSEDDSNU300I DSNURILD - (RE)LOAD PHASE COMPLETE, ELAPSED TIME=00:00:08 DSNU381I -DB2G DSNUGSRX - TABLESPACE PAOLOR3.CLOBBTS1 IS IN COPY PENDING DSNU381I -DB2G DSNUGSRX - TABLESPACE PAOLOR3.CLOBATS1 IS IN COPY PENDING DSNU010I DSNUGBAC - UTILITY EXECUTION COMPLETE, HIGHEST RETURN CODE=4

6.2.2 DSNTIAULIf you select a LOB column in a list of field specifications when unloading a table, or select a LOB column by default (by omitting a list of field specifications), LOB data is materialized in the output, if the definition of the field is less than or equal to (<=) 32 KB. Any column greater than (>) 32 KB cannot be unloaded using this program. DSNTIAUL returns truncated data.

Unloading data from an image copy data set is not available for LOB columns.

DSNTIAUL works fine for unloading tables containing LOB columns with a size less than (<) 32 KB as shown in Example 6-5.

Example 6-5 Output from Unload with LOB column < 32 KB

DSN RUN PROGRAM(DSNTIAUL) PLAN(DSNTIB71) DSN END READY END DSNT490I SAMPLE DATA UNLOAD PROGRAM DSNT503I UNLOAD DATA SET SYSPUNCH RECORD LENGTH SET TO 80 DSNT504I UNLOAD DATA SET SYSPUNCH BLOCK SIZE SET TO 27920 DSNT503I UNLOAD DATA SET SYSREC00 RECORD LENGTH SET TO 87 DSNT504I UNLOAD DATA SET SYSREC00 BLOCK SIZE SET TO 27927 DSNT495I SUCCESSFUL UNLOAD 21 ROWS OF TABLE "PAOLOR3"."CLOB_BASE_TABLE

When using DSNTIAUL on LOB tables with columns > 32 KB, the output is truncated to about 32 KB. Below in Example 6-6 is the output from an unload of a LOB table with rows > 32 KB.

Example 6-6 Output from DSNTIAUL where column length > 32 KB

DSNT490I SAMPLE DATA UNLOAD PROGRAM DSNT503I UNLOAD DATA SET SYSPUNCH RECORD LENGTH SET TO 80 DSNT504I UNLOAD DATA SET SYSPUNCH BLOCK SIZE SET TO 27920 WARNING: DATA FROM COLUMN 4 WAS TRUNCATED TO 32756 BYTES FROM 2693248. WARNING: DATA FROM COLUMN 4 WAS TRUNCATED TO 32668 BYTES FROM 32756. DSNT503I UNLOAD DATA SET SYSREC00 RECORD LENGTH SET TO 32756 DSNT504I UNLOAD DATA SET SYSREC00 BLOCK SIZE SET TO 32756 DSNT495I SUCCESSFUL UNLOAD 1 ROWS OF TABLE "PAOLO"."CLOB_BASE_TABLE"

6.2.3 UNLOADThe UNLOAD utility was introduced in DB2 V7. It can provide better performance, enhanced functionality, higher availability, improved concurrency, and low cost of operation, compared to previous methods used for unloading data.


As we already have mentioned for DSNTIAUL, the UNLOAD utility does not act against LOB tables, because they are, in general, not available for access via SQL nor the UNLOAD utility. Every statement referring to LOB values has to be issued against the base table.

You can use the UNLOAD utility for unloading your LOB data, but the maximum size is also represented by the maximum record length of your output file. When you want to truncate your data to fit in your output data set, you can use the syntax in Example 6-7.

Example 6-7 Using UNLOAD with truncation

UNLOAD TABLESPACE LOBDB.BASETSFROM BOOK_BASE_TABLE(BOOK_TEXT CLOB (10000) TRUNCATE)

The TRUNCATE keyword tells DB2 to truncate after the given number of bytes. If you are dealing with BLOBs, truncation of BLOB values normally results in unusable data in the case of images, movies or audio data. This data truncation occurs only when you explicitly specify the TRUNCATE option, this prevents you from being unaware of possible data truncation.

If you do not specify the truncation option for your LOB data, the message reported in Example 6-8 is issued by DB2 whenever your composite record size is greater than 32 KB.

Example 6-8 Using the UNLOAD utility on a base table with row size > 32 KB

DSNU000I DSNUGUTC - OUTPUT START FOR UTILITY, UTILID = UNLOADDSNU050I DSNUGUTC - UNLOAD TABLESPACE PAOLO.CLOBBTS DSNU1218I -DB2G DSNUULIA - LOGICAL RECORD LENGTH OF OUTPUT RECORD EXCEEDED THE LIMIT FOR TABLE PAOLO.CLOB_BASE_TABLEDSNU012I DSNUGBAC - UTILITY EXECUTION TERMINATED, HIGHEST RETURN CODE=8

When you specify BLOB, CLOB or DBCLOB as a field type, a four byte binary length field is placed in the output record prior to the actual data field. If your LOB column is nullable, a NULL indicator byte is placed before the length field.

Any CCSID conversion is applied first, before the data is considered for encoding in output. For CLOBs, no conversion is applied if the subtype is BIT. For more information on data truncation and conversion purposes, refer to DB2 UDB for OS/390 and z/OS Version 7 Utility Guide and Reference, SC26-9945.

The UNLOAD utility does not support full image copies of LOB table spaces in input.

6.2.4 COPY Both full and incremental image copies are supported for a LOB table space, as well as SHRLEVEL REFERENCE, SHRLEVEL CHANGE, and the CONCURRENT options. COPY with CONCURRENT option copies empty or unformatted allocated data pages for a LOB table space.

You can quiesce and copy both the base table space and the LOB table space at the same time to establish a recoverable point of consistency. Be aware that QUIESCE does not create a recoverable point for a LOB table space that contains LOBs defined with LOG NO.

Another way is to run the COPY UTILITY for the base table space and the LOB table space in one UTILITY control statement as listed in Example 6-9.


Example 6-9 Copy of base and LOB table space in one statement

COPY TABLESPACE PAOLO.CLOBATS COPYDDN SYSREC00 FULL YES TABLESPACE PAOLO.CLOBBTS COPYDDN SYSREC01 FULL YES SHRLEVEL REFERENCE DSNU050I DSNUGUTC - COPY TABLESPACE PAOLO.CLOBATS COPYDDN SYSREC00 FULL YESTABLESPACE PAOLO.CLOBBTS COPYDDN SYSREC01 FULL YES SHRLEVEL REFERENCE DSNU400I DSNUBBID - COPY PROCESSED FOR TABLESPACE PAOLO.CLOBATS NUMBER OF PAGES=54 AVERAGE PERCENT FREE SPACE PER PAGE = 2.90 PERCENT OF CHANGED PAGES = 40.90 ELAPSED TIME=00:00:00 DSNU400I DSNUBBID - COPY PROCESSED FOR TABLESPACE PAOLO.CLOBBTS NUMBER OF PAGES=3 AVERAGE PERCENT FREE SPACE PER PAGE = 35.33 PERCENT OF CHANGED PAGES = 3.57 ELAPSED TIME=00:00:00 DSNU428I DSNUBBID - DB2 IMAGE COPY SUCCESSFUL FOR TABLESPACE PAOLO.CLOBATS DSNU428I DSNUBBID - DB2 IMAGE COPY SUCCESSFUL FOR TABLESPACE PAOLO.CLOBBTS DSNU010I DSNUGBAC - UTILITY EXECUTION COMPLETE, HIGHEST RETURN CODE=0

This will create a single RBA in SYSIBM.SYSCOPY for recovery purposes. Querying the SYSCOPY table you get the common RBA recorded with the above image copy as shown in Example 6-10.

Example 6-10 Query of SYSCOPY for common RBA

SELECT DBNAME,TSNAME,ICTYPE,ICDATE,ICTIME,HEX(START_RBA) FROM SYSIBM.SYSCOPY WHERE DBNAME = 'PAOLO' AND TSNAME LIKE 'CL%' ---------+---------+---------+---------+---------+---------+---------+---------+DBNAME TSNAME ICTYPE ICDATE ICTIME COL1---------+---------+---------+---------+---------+---------+---------+---------+PAOLO CLOBATS F 020319 142508 0002A9A86268 PAOLO CLOBBTS F 020319 142509 0002A9A86268 DSNE610I NUMBER OF ROWS DISPLAYED IS 2 DSNE616I STATEMENT EXECUTION WAS SUCCESSFUL, SQLCODE IS 100

If you take an inline image copy of a table with LOB columns, DB2 makes a copy of the base table space, but does not copy the LOB table spaces.

The SHRLEVEL optionsIn general, when you use SHRLEVEL CHANGE for image copies of your LOB table space, somebody can interfere with your job and manipulate the content of the table you are currently backing up. Consider the following example: an image copy job backs up your LOB table space and it takes a couple of minutes, depending on the number and size of your LOBs stored in the auxiliary table. When somebody inserts a LOB value, some pages can be placed in the already backed up area of your table space while other pages can be stored in an area of your LOB table space your job has not backed up yet. When you have used LOG YES for the definition of the LOB table space, you can acquire a quiesce point after the image copy is taken to create a recoverable point in time. But when you have specified LOG NO for your LOB table space, you can be in trouble if you have to recover to an image copy taken with SHRLEVEL CHANGE.

This means that you are not protected against an inconsistent status of your image copy when using SHRLEVEL CHANGE. We recommend using SHRLEVEL REFERENCE for copying the LOB table space, regardless if you take incremental or full image copies.


6.2.5 REPORT TABLESPACESETThe REPORT TABLESPACESET utility reports the LOB table spaces as part of the table space set that includes the base table space. So because of that REPORT TABLESPACESET can be used to identify the LOB table spaces that were implicitly created by DB2 when SQLRULES(STD) is used, as you can see in Example 6-11.

Example 6-11 REPORT TABLESPACESET with CURRENT RULES STD created LOBs

DSNU000I DSNUGUTC - OUTPUT START FOR UTILITY, UTILID = PAOLOR20DSNU050I DSNUGUTC - REPORT TABLESPACESET TABLESPACE PAOLO.CLOBBTS DSNU587I -DB2G DSNUPSET - REPORT TABLESPACE SET WITH TABLESPACE PAOLO.CLOBBTS TABLESPACE TABLE DEPENDENT TABLE INDEXSPACE INDEX PAOLO.CLOBBTS PAOLO.CLOB_BASE_TABLE

BASE TABLESPACE LOB TABLESPACE AUXILIARY TABLE INDEXSPACE INDEX PAOLO.CLOBBTS PAOLO.L1LX$XO8 PAOLO.CLOB_DOCUM1LXA0LID PAOLO.ICLOBRDO PAOLO.ICLOB_DOCUM1LXARNXDSNU580I DSNUPORT - REPORT UTILITY COMPLETE - ELAPSED TIME=00:00:00 DSNU010I DSNUGBAC - UTILITY EXECUTION COMPLETE, HIGHEST RETURN CODE=0

6.2.6 RUNSTATSWhen dealing with LOBs, you can run the RUNSTATS utility against the base table space, the LOB table space, or the index on the auxiliary table.

RUNSTATS on the base table spaceIn SYSIBM.SYSCOLUMNS, the cardinality of a LOB column, COLCARDF, is determined by counting only non-null and non-zero length columns. HIGH2KEY and LOW2KEY are not applicable for the LOB column and will contain blanks.

In SYSIBM.SYSCOLSTATS, the cardinality of a LOB column, COLCARDF, is determined by counting only non-null and non-zero length columns. HIGHKEY, LOWKEY, HIGH2KEY and LOW2KEY are not applicable for the LOB column and will contain blanks.

RUNSTATS on the index o the auxiliary tableThe statistics, NEAROFFPOSF and FAROFFPOSF in SYSIBM.SYSINDEXPART are not applicable for the index on the auxiliary table. For an index on the auxiliary table, the CARDF column of SYSIBM.SYSINDEXPART indicates the number of keys in the index which refer to LOBs. The statistics CLUSTERED and CLUSTERRATIO in SYSIBM.SYSINDEXPART are not applicable for the index on the auxiliary table.

RUNSTATS on the LOB table spaceRUNSTATS support is extended to LOB table spaces. The statistics gathered for a LOB table space are used to determine whether or not a LOB table space needs to be REORGed. The statistics in SYSIBM.SYSTABLESPACE will be updated. Statistics in SYSIBM.SYSTABLES will not be updated except for CARDF which contains the number of LOBs in the auxiliary table. Statistics in SYSIBM.SYSTABLEPART will not be updated except for CARD which will contain the number of LOBs. PERCACTIVE will contain -2 to indicate that this field is not updated for auxiliary tables. Statistics in SYSIBM.SYSCOLUMNS will not be updated. HIGH2KEY and LOW2KEY will contain blanks for the columns of the auxiliary table, COLCARDF will contain -2 to indicate that this field is not updated for auxiliary tables. The statistics for a LOB table space will be stored in a new catalog table called SYSIBM.SYSLOBSTATS. Its columns are described in Table 6-2.


Table 6-2 SYSIBM.SYSLOBSTATS columns

Only the keywords in Table 6-3 are allowed if the table space specified by the TABLESPACE keyword is a LOB table space.

Table 6-3 Allowed keywords and options

6.2.7 REORGThe REORG utility for LOB table spaces is different from the normal table space REORG utility.

In fact a REORG of a LOB table space is not compatible with any other utility: the LOB table space is unavailable to other applications during REORG processing.

For REORG on non-LOB table spaces you may use OFFPOSLIMIT or INDREFLIMIT catalog query options to decide when to trigger your Reorg. If you specify the REPORTONLY option, REORG will produce a report detailing if a REORG is recommended; a REORG will not be performed. You cannot run REORG with the keyword REPORTONLY on a LOB table space.

While LOAD allows LOG YES or NO for LOBs, REORG requires LOG YES.

To get information on when to run a REORG on LOB table spaces you can view SYSIBM.SYSLOBSTATS. This table contains information about how the data in the table space is physically stored. The pertinent columns are AVGSIZE, FREESPACE, and ORGRATIO.

Running RUNSTATS updates the SYSLOBSTATS table. If you then select from it, you get results similar to those shown in Example 6-12.

Column name Description

AVGSIZE Average size of a LOB in a LOB table space.

FREESPACE Amount of free space in the LOB table space.

ORGRATIO The ratio of organization in the LOB table space. A value of 1 indicates perfect organization of the LOB table space. The greater than 1 the value, the more disorganized is the LOB table space.

DBNAME Database that contains the LOB table space named in NAME.

NAME Name of the LOB table space.

STATSTIME The time that RUNSTATS was executed.

Keyword Valid options

SHRLEVEL REFERENCE / CHANGE

REPORT YES / NO

UPDATE ALL / NONE

Note: Apply the PTF for PQ42864 to allow RUNSTATS on a LOB table space to produce correct statistics values for FREESPACE, ORGRATIO, PERCACTIVE, and PCTROWCOMP.


Example 6-12 Table SYSIBM.SYSLOBSTATS

L STATSTIME AVGSIZE FREESPACE ORGRATIO DBNAME NAME * * * * * * - -------------------------- ----------- ----------- -------- -------- --------* 2001-07-13-21.36.18.835699 0 156 0.00 DSN8D71L DSN8S71M 2001-07-13-21.36.19.107433 1210 140 2.00 DSN8D71L DSN8S71N 2001-07-13-21.36.18.536620 0 156 0.00 DSN8D71L DSN8S71L******************************* END OF DB2 DATA *******************************

The definitions of the columns are:

� AVGSIZE

AVGSIZE is the average size of all LOBs in the table space, or the total number of LOB bytes divided by the number of LOBs. For example, if you had an employee column for a picture of the employee saved as a LOB, the format used (GIF, JPEG, and so on) will affect the size. The typical value of size within the column is indicated by AVGSIZE. AVGSIZE does not contribute to triggering of any utility. Instead it can be used to estimate the data set sizes of work and sort data sets in LOB data set management.

� FREESPACE

This value has no relationship with the FREESPACE parameter in the CREATE TABLESPACE statement. In this case FREESPACE is the number of kilobytes of available space in the LOB table space as reflected in the space maps, and it provides an indication of how many more LOBs can be added in the existing extents already allocated.

� ORGRATIO

ORGRATIO is a measure of fragmentation or non-optimal organization of the LOBs stored in the LOB table space. A value of 1.0 is optimal. The more this value exceeds 1, the more your LOB table space is disorganized.

More information on ORGRATIOA LOB column always has an auxiliary index which locates the LOB within the LOB table space. Access path is not an issue, because LOB access is always done via an index probe using the auxiliary index. However, performance can be affected if LOBs are scattered into more physical pieces than necessary, thus involving more I/O to materialize.

As you can see in Figure 6-1, there are four LOBs accessed via the auxiliary index. These LOBs are stored in chunks of pages. A chunk consists of 16 contiguous pages of LOB data. If the size of a LOB is smaller than a chunk (smaller than 16 pages), then it is expected to fit in one chunk. If the size is greater than a chunk (greater than 16 pages), then it is optimized to fit into the minimum number of chunks (LOB size divided by chunk size). If, because of fragmentation within chunks, LOBs are split up to store in more chunks than optimal, the ORGRATIO will increase.

The small e at the bottom of a chunk in Figure 6-1 marks the end of a single LOB value.


Figure 6-1 Fragmented LOB table space

In our example, the first part of LOB 1 (accessed by rowid 1) is stored in chunk 1, the remaining bytes are placed in data pages of chunk number 2. Because of its size, LOB number 1 should fit into one chunk, but two chunks are used to store its value. This means that one extra chunk is used to store LOB number 1. Because of its size, LOB number 2 can at least be stored in two chunks which are also used by DB2 to store it. So there are no extra chunks for LOB number 2. The value of LOB number 3 should fit into one chunk and is also stored in one chunk which means that there are no extra chunks used for its storage. Things look different when you look at LOB number 4. The value behind LOB number 4 is expected to fit in two chunks, but DB2 uses three chunks. So there is another extra chunk used for LOB number 4.

DB2 uses the following formula to calculate an appropriate value of ORGRATIO:

ORGRATIO = (� extra chunks + � LOBs) / � LOBs

If a LOB should fit in a number of n chunks, but n + m chunks are used to store its value, the number of m is referred to as extra chunks.

After reorganization of the LOB space, Figure 6-2, the LOBs are placed in the optimal number of chunks. Since there are no extra chunks allocated, the ORGRATIO is 1.0 according to the formula shown below.

LOBs ORGRATIO - fragmented

No access path statistics for LOBsORGRATIO = (#extra chunk LOBs + #LOBs) / #LOBs = (2 + 4) / 4, or 1.5

rowid 1

rowid 4rowid 3rowid 2

chunk 1 chunk 2 chunk 3 chunk 4 chunk 5

Auxiliary index

Auxiliary table (LOB table space)

E E EE


Figure 6-2 Non-fragmented LOB table space

Running REORGRunning the REORG table space utility on a LOB table space will help increase the effectiveness of prefetch. For a LOB table space, REORG table space performs these actions:

� Removes embedded free space � Attempts to make LOB pages contiguous

Consider running REORG on the LOB table space when the value in the ORGRATIO column of SYSLOBSTATS is 2 or higher. The value is set by running the RUNSTATS utility. After inserting LOBs into your LOB table spaces and running the RUNSTATS utility, you may see values in ORGRATIO like 40.00 or 100.00. This depends on the size of the LOBs, and how many space map pages they span; small LOBs tend to show the ORGRATIO of 1. Running the REORG utility on these LOB table spaces that have been inserted with big LOBS will reset the value. Running the RUNSTATS utility on the ‘big’ reorged LOB table space will show the desired 1.00.

REORG on a LOB table space will free embedded space and will attempt to make the LOB PAGES contiguous. Reorg will not do a delete and define for the LOB table space itself. It will not get rid of the empty data sets.

If you want to adjust the primary allocation or to get rid of secondary extents, you can use RECOVER. That is, you change PRIQTY and SECQTY, then run RECOVER , and RECOVER will do a delete/define. However RECOVER will not get rid of secondary data sets. The only way to get rid of A002, A003, ... is by using an explicit drop/recreate of the object.

If you use REORG SHRLEVEL NONE on a LOB table space and the LOB Manager determines that nothing needs to be done to the table space, no COPY pending status is set.

LOBs ORGRATIO - after REORG

Improved performance after REORGORGRATIO = (#extra chunk LOBs + #LOBs) / #LOBs = (0 + 4) / 4, or 1.0

rowid 1

rowid 4rowid 3rowid 2

chunk 1 chunk 2 chunk 3 chunk 4 chunk 5

Auxiliary index

Auxiliary table (LOB table space)

E EEE


However, if the LOB Manager indicates that changes are needed, REORG places the reorganized LOB table space or partition in COPY pending status. In this situation, perform a full image copy to reset the COPY pending status and to ensure that a backup is available for recovery.

Reclaim space for LOB table spacesThe FREESPACE gives you an indication of how much allocated space is available for more LOBs. LOB table spaces can accumulate dead space that can be reclaimed during reorganization. If this dead space is not regularly reclaimed, it may result in acquiring more extents than needed to add additional LOBs.

For LOB table spaces, an updated LOB will be written without reclaiming the old version of the LOB immediately.

When FREESPACE approaches zero for a LOB table space, it is a good time to reclaim the space, but there are no direct statistics indicating how much will be reclaimed. You can calculate the percentage of free space for LOB table spaces based on the values of FREESPACE in SYSLOBSTATS and SPACEF in SYSTABLEPART:

(FREESPACE / SPACEF) * 100

We recommend that you reorganize LOB table spaces, if the percent of free space is less than 10%.

REORG does honor the PCTFREE and FREEPAGE values specified at CREATE TABLESPACE time, but they do not make much sense with LOBs, and should be set to zero. REORG will not delete and redefine the data set during reorganization of a LOB table space. If the number of extents are a problem, they can be changed, if DB2 managed, by altering the PRIQTY and SECQTY values, and then running RECOVER TOCOPY to delete and redefine the data sets with the new space quantities.

REORG UNLOAD EXTERNALThe UNLOAD EXTERNAL option is not available for LOB table spaces and its use will result in an error message. When you try to use this option for a REORG on a base table, which is possible, it depends on the declared size of your LOB values if DB2 lets you do it or not. As long as the composite size of your columns does not exceed the maximum record length, you are allowed to do it. Otherwise the message shown in Example 6-13 is issued.

Example 6-13 Trying to use REORG UNLOAD EXTERNAL on a base table row > 32 KB

DSNU000I DSNUGUTC - OUTPUT START FOR UTILITY, UTILID = REORG DSNU050I DSNUGUTC - REORG TABLESPACE PAOLO.CLOBBTS UNLOAD EXTERNAL DSNU297I -DB2G DSNURFTB - COMPOSITE RECORD SIZE TOO LARGE FOR PAOLO.CLOB_BASE_TABLEDSNU012I DSNUGBAC - UTILITY EXECUTION TERMINATED, HIGHEST RETURN CODE=8

6.2.8 CHECK LOBThe CHECK LOB online utility can be run against a LOB table space to identify any structural defects in the LOB table space and any invalid LOB values.

You must first recover a LOB table space that is in RECOVER-pending status before running CHECK LOB.

If you plan to run CHECK DATA on a base table space containing at least one LOB column, complete the following steps prior to running CHECK DATA:


1. Run CHECK LOB on the LOB table space.

2. Run CHECK INDEX on the index on the auxiliary table prior to running CHECK DATA to ensure the validity of the LOB table space and index on the auxiliary table.

3. Run CHECK INDEX on the base table space indexes.

If the LOB table space is in either the CHECK-pending or RECOVER-pending status, or if the index on the auxiliary table is in REBUILD-pending status, CHECK DATA will issue an error message and fail.

Run the CHECK LOB online utility against a LOB table space that is marked CHECK pending (CHKP) to identify structural defects. If none is found, the CHECK LOB utility turns the CHKP status off.

Run the CHECK LOB online utility against a LOB table space that is in auxiliary warning (AUXW) status to identify invalid LOBs. If none exists, the CHECK LOB utility turns AUXW status off.

Run CHECK LOB after a conditional restart or a point-in-time recovery on all table spaces where LOB table spaces may not be synchronized.

After successful execution, CHECK LOB resets the CHECK pending (CHKP) and auxiliary warning (AUXW) statuses.

6.2.9 CHECK DATAIf you run CHECK DATA AUXERROR REPORT or INVALIDATE on a base table space containing at least one LOB column, the following errors may be reported:

� Orphan LOBs

An orphan LOB column is a LOB found in the LOB table space but not referenced by the base table space. An orphan can result if you recover the base table space to a point in time prior to the insertion of the base table row or prior to the definition of the LOB column. An orphan can also result if you recover the LOB table space to a point in time prior to the deletion of a base table row.

� Missing LOBs

A missing LOB column is a LOB referenced by the base table space, but the LOB is not in the LOB table space. A missing LOB can result if you recover the LOB table space to a point in time when the LOB column is not in the LOB table space. This could be a point in time prior to the first insertion of the LOB into the base table, or when the LOB column is null or has a zero length.

� Out-of-synch LOBs

An out-of-synch LOB error occurs when DB2 detects a LOB that is found in both the base table and the LOB table space, but the LOB in the LOB table space is at a different level. A LOB column is also out-of-synch if the base table LOB column is null or has a zero length, but the LOB is found in the LOB table space. An out-of-synch LOB can occur anytime you recover the LOB table space or the base table space to a prior point in time.

� Invalid LOBs

An invalid LOB is an uncorrected LOB column error found by a previous execution of CHECK DATA AUXERROR INVALIDATE.

Note: Apply PTF UQ64673 for APAR PQ46137 to eliminate several errors while executing CHECK DATA.


Depending on the AUXERROR clause specified, the following actions are performed:

� With AUXERROR REPORT

DB2 sets the base table space to the auxiliary CHECK-pending (ACHKP) status. If CHECK DATA encounters only invalid LOB columns and no other LOB column errors, the base table space is set to the auxiliary warning status.

� With AUXERROR INVALIDATE

DB2 sets the base table LOB column to an invalid status, and sets the base table space to the auxiliary warning (AUXW) status. You can use SQL to update a LOB column in the AUXW status, however, any other attempt to access the column will result in a -904 SQL return code.

If you use exception tables, the exception table for the base table must have a similar LOB column and a LOB table space for each LOB column. If an exception is found, DB2 moves the base table row with its LOB column to the exception table, and moves the LOB column into the exception table's LOB table space. If you specify DELETE YES, DB2 deletes the base table row and the LOB column.

An auxiliary table cannot be an exception table. A LOB column check error is not included in the exception count. A row with a LOB column check error only does not participate in exception processing.

You must complete all LOB column definitions for a base table before running CHECK DATA. A LOB column definition is not complete until the LOB table space, auxiliary table and index on the auxiliary table have been created. If any LOB column definition is not complete, CHECK DATA will fail and issue error message DSNU075E.

If you terminate CHECK LOB during the CHECKLOB phase, it sets the table space to CHECK-pending status. Otherwise, CHECK LOB resets the CHECK-pending status at the end of the phase if no errors are detected.

You can restart a CHECK LOB utility job, but it starts from the beginning again.

Auxiliary warning (AUXW) status is set on when at least one base table LOB column has an invalidated LOB as a result of running CHECK DATA AUXERROR INVALIDATE. An attempt to retrieve an invalidated LOB results in a -904 SQL return code.

The RECOVER utility also sets AUXW status if it finds an invalid LOB column. Invalid LOB columns may result from the following actions (all three must apply):

� LOB table space was defined with LOG NO. � LOB table space was recovered. � LOB was updated since the last image copy.

If you run CHECK LOB, and LOB table space errors are found, the table space is placed in CHECK-pending status.

Complete the following tasks to remove the CHECK-pending status:

� Correct any defects found in the LOB table space either through SQL update/delete, or by using the REPAIR utility.

� Run CHECK LOB again, or run the REPAIR utility, in order to reset CHECK-pending or AUXW status.

Table 6-4 summarizes the actions for resetting the AUXW status.


Table 6-4 Resetting AUXW flags

CHECK LOB issues message DSNU743I whenever a LOB value is invalid. The violation is identified by:

� The row ID and version number of the LOB � A reason code for the error � The page number where the error was found

You can resolve LOB violations by using the UPDATE or DELETE SQL statements to update the LOB column or delete the row associated with the LOB (use the rowid given in message DSNU743I).

If CHECK LOB issues either message DSNU785I or DSNU787I, it has detected a logical inconsistency within the LOB table space. Contact IBM Support Center for assistance with diagnosing and resolving the problem.

Table 6-5 summarizes the actions to reset the ACHKP status.

Table 6-5 Resetting AUX CHECK pending status

Resetting auxiliary warning status

Status Abbrevia-tion

Object Affected

Corrective Action Notes

Auxiliary Warning

AUXW Base table space

1) Update or delete invalid LOBs through SQL2) Run CHECK DATA utility to verify the validity of LOBs and reset the AUXW status.

1, 2, 3

LOB table space

1) Update or delete invalid LOBS through SQL.2) Run CHECK LOB utility to verify the validity of LOBs and reset the AUXW Status.Alternatively you can run the REPAIR utility to set NOLOBCHKP.Be aware that using the REPAIR utility to delete invalid LOBS may cause the base tables and the Index on the AUX table to reference invalid LOBS.

1

Notes

1) A base table space or LOB table space in the AUXW status is available for processing by SQL, even though it contains invalid LOBS However, an attempt to retrieve an invalid LOB returns a -904.

2) DB2 can access all rows of a base table space that is in the AUXW status. SQL can update the invalid LOB column and delete base table rows, but the value of the LOB column cannot be retrieved. If DB2 attempts to access an invalid LOB column, a -904 SQL code is returned. The AUXW status remains on the base table space even when SQL deletes or updates the last invalid LOB column

3) If CHECK DATA AUXERROR REPORT encounters only invalid LOB columns and no other LOB column errors, the base table space is set to auxiliary warning status.

Resetting auxiliary check-pending status

Status Abbreviation Object affected Corrective Action Notes

Auxiliary Check Pending

ACHKP Base table space

Run the Check Data utility with the Appropriate scope option to verify the validity of LOBS and reset ACHKP status.

1)

Notes

1) A base table space in the ACHKP status is unavailable for processing by SQL.


6.2.10 CHECK INDEXWhen checking an auxiliary table index, CHECK INDEX verifies that each LOB is represented by an index entry.

To run this utility, the privilege set of this process must include one of the following authorities:

� STATS privilege for the database � DBADM, DBCTRL, or DBMAINT authority for the database � SYSCTRL or SYSADM authority

An ID with installation SYSOPR authority can also execute CHECK LOB.

6.2.11 DSN1COPYThe standalone utility DSN1COPY works on LOB table spaces just as with other types, with the exception that, when working with LOB table spaces, the parm field LOB must be specified.

LOB specifies that the SYSUT1 data set is a LOB table space. Empty pages in the table space are copied, but no error messages are issued. You cannot specify the SEGMENT and INLCOPY options with the LOB parameter.

DB2 attempts to determine if the input data set is a LOB data set. If you specify the LOB option but the data set is not a LOB data set, or if you omit the LOB option but the data set is a LOB data set, DB2 issues an error message and terminates.

Example 6-14 shows the error you receive when not using the LOB parameter for a LOB table space.

Example 6-14 DSN1COPY error for LOB table space

DSN1999I START OF DSN1COPY FOR JOB PAOLOR3C PH01S01 DSN1954I DSN1COPY PARAMETER PAGESIZE OR LOB IS MISSING OR INCORRECTLY SPECIFIEDDSN1993I DSN1COPY TERMINATED, 0 PAGES PROCESSED

We used DSN1COPY to restore data to a different base and auxiliary table from a full image copy data set. The JCL for the LOB table space DSN1COPY restore is reported in Example 6-15.

Example 6-15 JCL for DSN1COPY OBIDXLAT for LOB table space

//PH01S01 EXEC PGM=DSN1COPY, // PARM=('FULLCOPY,RESET,LOB,32K,OBIDXLAT') //STEPLIB DD DSN=DB2G7.SDSNLOAD,DISP=SHR //SYSPRINT DD SYSOUT=* //SYSUDUMP DD SYSOUT=* //SYSUT1 DD DSN=PAOLOR3.IMAGE.CLOBATS1.IC0322,DISP=OLD //SYSUT2 DD DSN=DB2V710G.DSNDBC.PAOLOR3.CLOBATS2.I0001.A001,DISP=OLD//SYSXLAT DD * 307 307 5 14 6 15 ***********************************************************************************DSN1COPY IS PROCESSED WITH THE FOLLOWING OPTIONS: NO CHECK/NO PRINT/32K/FULLCOPY /NON-SEGMENT/NUMPARTS = 0/ OBIDXLAT/NO VALUE/ RESET/ /LOB/PIECESIZ= / DSSIZE= INPUT DSNAME = PAOLOR3.IMAGE.CLOBATS1.IC0322 , SEQ


OUTPUT DSNAME = DB2V710G.DSNDBC.PAOLOR3.CLOBATS2.I0001.A001 , VSAM DSN1COPY COMPLETED SUCCESSFULLY, 00000041 PAGES PROCESSED

6.2.12 REPAIRWhen dealing with LOBs, the LOCATE table space statement locates data to be repaired within a table space.

When you specify LOCATE ROWID for a LOB table space, with the delete option, the LOB specified by ROWID is deleted with its index entry. All pages occupied by the LOB are converted to free space. The DELETE statement will not remove any reference to the deleted LOB from the base table space.

One LOCATE statement is required for each unit of data to be repaired. Several LOCATE statements can appear after each REPAIR statement.

With ROWID X'byte-string' you can specify that the data to be located is a LOB in a LOB table space. The byte-string is the row ID that identifies the LOB column.

The most likely source of a row ID is an orphaned LOB that is reported by the CHECK LOB utility. If you specify the ROWID keyword, the specified table space must be a LOB table space.

With VERSION X'byte-string' you can specify that the data to be located is a LOB in a LOB table space. The byte-string is the version number that identifies the version of the LOB column.

The most likely source of a version number is an orphaned LOB that is reported by the CHECK LOB utility, or an out-of-synch LOB that is reported by the CHECK DATA utility. If you specify the VERSION keyword, the specified table space must be a LOB table space.

6.2.13 RECOVERY In some ways, planning for LOB recovery is similar to that for user defined referential integrity since you have to remember that there is a relationship between a table with a LOB column and the associated LOB table space. It is true that tables involved in referential integrity relationships must be considered as part of the same table space set for recovery purposes. Similarly, a LOB table space and its associated base table space are part of a table space set. That set, too, is reported using REPORT table space.

Recovering to a prior point in timeYou can recover your LOB table space to a prior point in time, using TOLOGPOINT, TORBA, or TOCOPY. Ideally, you have taken copies of the LOB table space and the related base table space at a quiesce point. If you then recover to the quiesce point, you avoid having the LOB or base table space marked as check pending.

Thus, if the LOB table space was defined with LOG NO, any log records written for the LOB table space between its last image copy and the recover-to-point means the AUX WARNING will be turned on. Then you have to run the CHECK LOB utility to find any invalid LOBs. For more details see 6.3, “Recovery scenario” on page 111.


Recovering to the current point in timeRecovering to the current point in time for a base table containing a LOB column is not any different form the past. DB2 will apply the appropriate image copy to the base table space and read the log since copy time until present, redoing all the changes. This is also true for the LOB table space. However, if the LOB table space is defined with LOG NO and log records must be applied to the LOB, the LOB is invalid and DB2 will set AUX Warning (AUXW) for that LOB table space.

Recovering LOB PAGES on the Logical Page ListIf there are LPL entries for a LOB table space the same procedure has to take place as for regular table spaces (START DATABASE command with the SPACENAM option). But if the LOB table space is defined with LOG NO and that log data is needed, then LOB CHECK PENDING will be turned on for that table space. Therefore, you will have to run the CHECK LOB utility to identify which LOBs are invalid.

In generalYou should quiesce and copy both the base table space and the LOB table space at the same time to establish a recoverable point of consistency. Be aware that QUIESCE does not create a recoverable point for a LOB table space that contain LOBs defined with LOG NO.

The RECOVER utility sets AUXW status if it finds an invalid LOB column. Invalid LOB columns may result from the following actions (all three must apply):

� LOB table space was defined with LOG NO � LOB table space was recovered � LOB was updated since the last image copy

Object states after recoveryAfter describing and reviewing all the different utilities you may run in your LOB environment, the following table will give you a list of all the different states of the different objects you may receive when you are dealing with recovery scenarios in a LOB environment. See Table 6-6.

Table 6-6 Object status after different recovery scenarios

Determining object status after recovery

Object Recovery type Base table space status

Index on auxiliary table status

LOB table space status

Base table space Current RBA or LRSN none none none

Base table space Point in Time CHKP none none

Index on Auxiliary table

Current RBA or LRSN none none none

Index on Auxiliary table

Point in Time none CHKP none

LOB table space Current RBA or LRSN, LOB table space defined with LOG (YES)

none none none

LOB table space Current RBA or LRSN, LOB table space defined with LOG(NO)

none none AUXW (1)


6.3 Recovery scenarioIn the previous sections we have described the details and dependencies of using utilities regarding LOBs. As an example we will now walk you through a recovery scenario showing the steps to take for a successful recovery. This includes the LOB base table space, the LOB table space and the auxiliary index. Also it will give you an idea of what needs to be performed in similar situations. With this information you can preventively set up a scenario for your recovery case when dealing with LOBs.

Setting up the recovery environmentTo get started, we first implement the DB2 objects we need. How to create LOBs is described in detail in 3.2, “Storing LOBs” on page 16. For our purposes, we build a database (PAOLO), a base table space (CLOBBTS) with its base table (CLOB_BASE_TABLE), the LOB table space (CLOBATS) with the auxiliary table (CLOB_AUX_TABLE) and the auxiliary index (CLOBAIX) which goes with it. See Figure 6-3.

LOB table space TOCOPY, copy was taken with SHARELEVEL REFERENCE

CHKP RBDP none

LOB table space TOCOPY, copy was taken with SHARELEVEL CHANGE

CHKP RBDP CHKP, AUXW

LOB table space TORBA or TOLOGPOINT (not a quiesce point)

CHKP RBDP CHKP,AUXW

LOB table space TORBA or TOLOGPOINT (at a quiesce point)

CHKP RBDP none

Note: (1) If at any time a log record is applied to a LOB table space that results in a LOB being marked invalid, the LOB table space is set to AUXILIARY Warning Status.

Determining object status after recovery


Figure 6-3 Recovery environment

Creating a recovery situationIn order to recreate a reasonable recovery situation we recover to Copy the LOB table space (LOBATS), that was defined with LOG NO (this is recommended for performance reasons). However, after the last full image copy, LOBs have been accessed via insert and so the base table, the auxiliary table and the auxiliary index have been updated. We have used the last full image copy to recover the LOB table space to a previous point in time. See Figure 6-4.


Vala Valb ABCDEFG... . . .

Valc Vald 1234567... . . .

Base Table Space: CLOBBTS

Base TableCLOB_BASE_TABLE

Row ID

Row ID LOB Value


1234567... Picture 2

LOB Table Space:

CLOBATS

AuxiliaryIndex:

CLOBAIX

LOB Table

Data Base: PAOLO

CLOB_AUX_TABLE


Figure 6-4 Recovering the auxiliary table with full image copy

The recover has left the base table and index at a different point in time, compared to the auxiliary table. Because of the special relationship of base and auxiliary table, the row entries are now more on the base table and the auxiliary index than in the aux table, since inserts have occurred. A display on our database, after recovering the LOB table space, shows the status reported in Example 6-16.

Example 6-16 Display database after recovery, using last full image copy

DSNT360I -DB2G *********************************** DSNT361I -DB2G * DISPLAY DATABASE SUMMARY * GLOBAL DSNT360I -DB2G *********************************** DSNT362I -DB2G DATABASE = PAOLO STATUS = RW DBD LENGTH = 16142 DSNT397I -DB2G NAME TYPE PART STATUS PHYERRLO PHYERRHI CATALOG PIECE -------- ---- ---- ------------------ -------- -------- -------- ----- CLOBATS LS RW CLOBBTS TS RW,ACHKP,COPY CLOBAIX IX RW,RBDP ******* DISPLAY OF DATABASE PAOLO ENDED **********************

The display also gives us the information that the table space (CLOBBTS) is in aux check pending (ACHKP). This requests to run CHECK DATA on the auxiliary table. It also shows that the entire auxiliary index is in rebuild pending status (RBDP).



Valc Vald 1234567... . . .



Row ID

Row ID LOB Value


LOB Table Space:

CLOBATS

AuxiliaryIndex:

CLOBAIX

LOB Table

Data Base: PAOLO

CLOB_AUX_TABLE

Recover with afull image copy

Gone


Rebuilding the auxiliary indexFirst we focus on the auxiliary index. As the status indicated, we have to run a Rebuild Index utility (REBUILD INDEX (PAOLO.CLOBAIX)), in order to synchronize the index with the current contents of the auxiliary table. See Figure 6-5.

Figure 6-5 Rebuilding the index

After successfully running the rebuild index utility, the DISPLAY DATABASE command gives us the status information listed in Example 6-17.

Example 6-17 Display database after Rebuild Index

DSNT360I -DB2G *********************************** DSNT361I -DB2G * DISPLAY DATABASE SUMMARY * GLOBAL DSNT360I -DB2G *********************************** DSNT362I -DB2G DATABASE = PAOLO STATUS = RW DBD LENGTH = 16142 DSNT397I -DB2G NAME TYPE PART STATUS PHYERRLO PHYERRHI CATALOG PIECE -------- ---- ---- ------------------ -------- -------- -------- ----- CLOBATS LS RW CLOBBTS TS RW,ACHKP,COPY CLOBAIX IX RW ******* DISPLAY OF DATABASE PAOLO ENDED **********************


Row ID

Row ID LOB Value


LOB Table Space:CLOBATS

AuxiliaryIndex:

CLOBAIX

LOB Table

Data Base: PAOLO

CLOB_AUX_TABLE Rebuild

index utility



Valc Vald 1234567... . . .



Running the Check Data utilityAs you have seen, we still have the aux check pending (ACHKP,COPY) status left on the base table (CLOBBTS). We then run the CHECK DATA utility on the base table with SCOPE(AUXONLY) and AUXERROR(INVALIDATE). The scope keyword AUXONLY is used because only the base table needs checking. See the check data utility job output in Example 6-18.

Example 6-18 Check Data utility output

DSNUGUTC - OUTPUT START FOR UTILITY, UTILID = PAOLOR3 DSNUGUTC - CHECK DATA table space PAOLO.CLOBBTS SCOPE(AUXONLY) AUXERROR(INVALIDATE) SORTDEVT SYSDADSNUKDST - CHECKING TABLE PAOLO.CLOB_BASE_TABLE DSNUGSOR - SORT PHASE STATISTICS - NUMBER OF RECORDS=101 ELAPSED TIME=00:00:00 DSNUGSOR - SORT PHASE STATISTICS -

NUMBER OF RECORDS=1 ELAPSED TIME=00:00:00 DSNUKERK - TABLE=PAOLO.CLOB_BASE_TABLE COLUMN=DOCUMENT IS MISSING IN INDEX PAOLO.CLOBAIX ROWID=X'41C48DDCC1CED26F260401D37018010000000000032D' VERSION=X'0001' DSNUKDAT - CHECK TABLE PAOLO.CLOB_BASE_TABLE COMPLETE, ELAPSED TIME=00:00:00 -DB2G DSNUKRDN - TABLE=PAOLO.CLOB_BASE_TABLE COLUMN=DOCUMENT WAS SET INVALID ROWID=X'41C48DDCC1CED26F260401D37018010000000000032D' -DB2G DSNUGSRX - TABLESPACE PAOLO.CLOBBTS IS IN COPY PENDING -DB2G DSNUGSRX - TABLESPACE PAOLO.CLOBBTS IS IN AUX WARNING STATE DSNUK001 - CHECK DATA COMPLETE,ELAPSED TIME=00:00:02

DSNUGBAC - UTILITY EXECUTION COMPLETE, HIGHEST RETURN CODE=4

After running the Check Data utility, the status of the base table space (CLOBBTS) has changed from ACHKP,COPY to COPY,AUXW, as listed in Example 6-19. The CHECK DATA also gives us the information we need to clean up the base table row that still was pointing to an invalid LOB.

Example 6-19 Display database after Check Data

DSNT360I -DB2G *********************************** DSNT361I -DB2G * DISPLAY DATABASE SUMMARY * GLOBAL DSNT360I -DB2G *********************************** DSNT362I -DB2G DATABASE = PAOLO STATUS = RW DBD LENGTH = 16142 DSNT397I -DB2G NAME TYPE PART STATUS PHYERRLO PHYERRHI CATALOG PIECE -------- ---- ---- ------------------ -------- -------- -------- ----- CLOBATS LS RW CLOBBTS TS RW,COPY,AUXW CLOBAIX IX RW ******* DISPLAY OF DATABASE PAOLO ENDED **********************

Removing the copy pending and deleting the invalid ROWIDBefore we can delete the invalid ROWID, we have to run a full image copy to remove the copy pending flag on the BASE TABLE. Then the identified ROWID within the CHECK DATA output, as shown in Figure 6-6, can be deleted from the base table.


Figure 6-6 Delete row from base table

This is done by using normal SQL, pointing to the invalid ROWID in the SQL where clause and the statement look like:

DELETE FROM BASE_TABLEWHERE ROW_ID = ROWID(X'41C48DDCC1CED26F260401D37018010000000000032D’)

Solving the auxiliary warning on the base tableNow we still have to complete the recovery process by solving the AUXW status on the base table. This can be done running the CHECK DATA SCOPE(AUXONLY) AUXERR(REPORT) again, like we have done before in Example 6-18. After this execution, the display database shows that everything is in order again, as shown in Example 6-20.

Example 6-20 Display database at the end

DSNT360I -DB2G *********************************** DSNT361I -DB2G * DISPLAY DATABASE SUMMARY * GLOBAL DSNT360I -DB2G *********************************** DSNT362I -DB2G DATABASE = PAOLO STATUS = RW DBD LENGTH = 16142 DSNT397I -DB2G NAME TYPE PART STATUS PHYERRLO PHYERRHI CATALOG PIECE -------- ---- ---- ------------------ -------- -------- -------- ----- CLOBATS LS RW CLOBBTS TS RW CLOBAIX IX RW ******* DISPLAY OF DATABASE PAOLO ENDED **********************





Row ID

Row ID LOB Value



AuxiliaryIndex:

CLOBAIX

LOB Table

Data Base: PAOLO

CLOB_AUX_TABLE

SQL: delete from clob_base_table where row_id = rowid(x'.....')


Full image copy is reasonable with LOG NOThe last step we need to complete our recover scenario is to take a full image copy of the base and the auxiliary table. Because the LOB table was created with LOG(NO) only a full image copy with SHRLEVEL REFERENCE is the correct backup for the LOB environment. See Figure 6-7.

Figure 6-7 Take a full image copy

6.4 Operational scenarioThe most prevalent way to get LOB data onto the mainframe is through file transfer (FTP) from a non-host based platform and writing a program which runs on the host taking in this file to insert the data into the LOB table. Another possibility is using a program running on a non-host platform with file reference variables pointing to files on the workstation and using DRDA to insert it into the mainframe based database.

Another means is through the use of the DB2 Extenders, which use UDFs to accomplish the insert.

The extender model uses DRDA to connect workstations to the mainframe through a normal n-tier architecture.



..



Row ID

Row ID LOB Value



AuxiliaryIndex:

CLOBAIX

LOB Table

Data Base: PAOLO

CLOB_AUX_TABLE

FIC

Take a full image copy of the base table space

and the LOB table space

FIC


The evolution of n-tier architecture has demanded that data from the mainframe become available to workstations. You can enable a DRDA connection from the workstation to any member of the DB2 family using DB2 Connect. There are three flavors of DB2 Connect, PE (Personal Edition), EE (Enterprise Edition), or EE Unlimited Edition. DB2 Connect provides extensive application programming tools such as OLE DB, JDBC, SQLJ, DB2 CLI, and embedded SQL. DB2 Connect integrates with both Java and Microsoft models for developing Web applications.

When using DB2 Connect PE, it is built in the normal 2- tier approach. This may be desirable if the environment has native TCP/IP support available, and an intermediate server is not desirable.

When using DB2 connect in the EE configuration, it is a normal 3-tier architecture which connect LAN based workstations to the mainframe host databases. Some good uses for this are when the host system doesn't support native TCP/IP and needs support for JAVA APPLETS, or supporting workstations on platforms where PE cannot exist (UNIX).

The Unlimited edition of EE avoids the necessity of counting users which must happen when using EE. It is an MSU based licensing model, which lends itself well to Web based applications.

Backup considerationsData administration with LOBs is challenging, especially when you have to deal with LOG NO definitions on your LOB table spaces. DB2’s recovery process is built on log data sets by definition, but using large objects allows you to turn off logging mechanisms or even forces you to turn them off when you use them in a size bigger than 1 GB.

Backing up a high availability scenario having a LOG YES LOB table space can be compared with copying and recovering referential integrity tables from the logical point of view, except the amount of data is probably much higher for a single table space.

When your scenario requests high availability and you have to specify LOG NO for the LOB table space for whatever reasons and it is not necessary to apply updates during the day on your LOBs, you can consider collecting your LOB updates during the day and process them in your batch window. When you take a full image copy of your base table space and the LOB table space after your batch job is done, you are prepared again for the day. This helps you minimizing outage times during the day when you have to recover a LOG NO LOB table space.


Chapter 7. Performance with LOBs

In this chapter we discuss topics directly and indirectly related to LOBs performance. The chapter is structured as follows:

� LOB materialization� Buffer pools and group buffer pools� LOBs performance considerations

7


7.1 LOB materializationLOB materialization from the point of view of DB2 is the function DB2 uses to place a LOB value into contiguous storage in virtual storage. Materialization will happen also on disk, but we are especially interested in the cases where the LOB materializes in buffer pools, data spaces, or the user (allied) address space.

Where the LOB is materialized depends on the way the LOB data is retrieved. In general, because LOB values can be very large, DB2 will do the utmost to try to avoid the DB2 materialization of LOB data until it is absolutely necessary. DB2 will materialize the LOB in the following different cases:

� When your application calls a user-defined function with a LOB as an argument

� When it moves a LOB into or out of a stored procedure

� When it assigns a LOB host variable to a LOB locator host variable

� When it converts a LOB from one CCSID to another

With or without the use of a locator, retrieving a LOB value will always go through the buffer pools associated with the LOB table space. Basically, from the DB2 point of view, if a LOB needs to be materialized in storage, it will be placed in a data space.

The amount of storage used in data spaces for LOB materialization depends upon a number of factors. They include the size of the LOB, the number of LOBs in a statement that need to be materialized, and the use of cursors to hold position in an application. Figure 7-1 provides an overview of the materialization process.

Figure 7-1 DB2 materialization overview picture

DB2 DDF Address Space

StoredProcedure

Address Space

UserAddress Space

DB2 DBM1 Address Space

Local Buffer Pool

Data Spaces

Disk environment

Buf

fer M

anag

er

LO

B M

anag

er

Dat

a M

anag

er


The different cases of materializationIn order to give you a better picture of LOB materialization and when it will occur, we describe different scenarios. In general you can UPDATE, INSERT, DELETE, or SELECT a LOB value. As described in 4.2, “LOB locators” on page 46, there are good reasons for using LOCATORs when processing LOB values. Beside other benefits, a LOCATOR is recommended if you are careful about storage allocation within your users’ address space, overall performance, and useful handling your LOB data. Materialization within your DB2 data spaces depends on the way the LOB value is accessed.

SELECTFirst we assume you have two applications that will select a LOB for further processing. This could be printing a LOB that contains a book or storing it into a non DB2 data set. However, let us say we have application A using a LOCATOR and application B not using a LOCATOR, but rather a host variable. From the point of view of DB2 LOB materialization, application A and application B will not materialize the LOB in DB2 data spaces when they only select a LOB value. See Figure 7-2.

Figure 7-2 A SELECT application and LOB materialization

As you can see in these cases, application B may require the complete LOB data. Therefore the private user address space may require, based on the size of the LOB, a huge amount of storage within its user address space. Application A uses a locator and may just use the space it needs for the chunks of data pointed to through the defined locator.

U serAddress S pace

D B 2 D B M 1 Address S pac e

Lo ca l B uffer P oo l

D isk env iron m en t

B

uffe

r Man

ager

LO

B M

anag

er

Dat

a M

anag

er A

U serAddress S pace

B

Chapter 7. Performance with LOBs 121

In the case where you have applications within a distributed environment and if they are selecting LOB values via the network, involving the DB2 DDF address space, or if you are using STORED PROCEDURES, in all these cases the selected LOB data will not be materialized within a DB2 data space. They will internally use LOCATORs and move the data, in small chunks, from DISK through the buffer pool and the DDF address space, or the STORED PROCEDURES address space.

INSERTAssume a scenario where your application A and B INSERT into your LOB table space. From the point of view of materialization, it will show a different picture. Now there are data spaces involved. The materialization in DB2 data spaces will occur if you are using LOCATORs, and it will not if you do not use them. Still, not using LOCATORS, like in application B, will cause you to use more storage in your private user address space and again depending on the size of your LOB, this could be quite storage consuming. See Figure 7-3.

.

Figure 7-3 An INSERT application and LOB materialization

If your application will INSERT LOB values via the network (this implicates the use of DB2’s DDF address space), or it will use a STORED PROCEDURE, or converts from one CCSID to another, LOB materialization will occur in a DB2 data space.

DELETE and UPDATEAs in the SELECT case, the DELETE case will also not materialize the LOB value: instead it just de-allocates the pages where the LOB is stored. Because the UPDATE case implicitly will INSERT and DELETE, the materialization of the LOB within the data space is the same as with the INSERT case.

UserAddress Space

DB2 DBM1 Address Space

Local Buffer Pool

Data Spaces

Disk environment

Buf

fer M

anag

er

LO

B M

anag

er

Dat

a M

anag

er

UserAddress Space

A

B


7.1.1 Using data spaces for LOBsIf DB2 needs to materialize LOBs in storage they will be placed in a data space. So now we focus on the amount of storage used in data spaces for LOBs that need to be materialized. This depends upon a number of factors. They include the size of the LOBs, the quantity of LOBs in a statement that needs to be materialized, and if you use cursors to hold position in an application.

As mentioned, LOBs are stored in data spaces for various situations. Again this will appear when a LOB is a parameter of a UDF, when it is needed to be converted, or when passing LOBs in a distributed environment and also when it has to be moved into a stored procedure.

If you want to see the number of data spaces created by DB2 (for instance as a result of materialization), you can use the MVS console command:

D A,xxxxDBM1 (xxxx = subsystem id)

As you can see in Example 7-1, the MVS console command will give you the number and the name of the data spaces DB2 has created.

Example 7-1 MVS display command

IEE115I 15.39.29 2002.067 ACTIVITY 481 JOBS M/S TS USERS SYSAS INITS ACTIVE/MAX VTAM OAS 00005 00038 00008 00031 00024 00008/00030 00019 DB2GDBM1 DB2GDBM1 IEFPROC NSW S A=0194 PER=NO SMC=000 PGN=N/A DMN=N/A AFF=NONE CT=223.764S ET=67.22.51 WUID=STC02493 USERID=STC WKL=SYSTEM SCL=SYSSTC P=1 RGP=N/A SRVR=NO QSC=NO ADDR SPACE ASTE=03EAB500 DSPNAME=DB2GD000 ASTE=4DA3AF80 DSPNAME=DB2GD001 ASTE=005C4080 DSPNAME=DB2GD002 ASTE=005C4100

If you may want to monitor the maximum storage used for LOBs in a data space, there is the IFCID 2 record for accounting and the IFCID 3 record for statistics. Both records contain the field QXSTLOBV, that provides the values.

If you have activated the statistics trace with IFCID 2 and the accounting traces with IFCID 3 in your DB2 subsystem, you will find the values within DB2 PM accounting and statistics record printout. In the report you will find the values in the miscellaneous part. The values shown there are in kilobytes as you can see in the Example 7-2 printout.

Example 7-2 DB2 PM accounting miscellaneous values

MISCELLANEOUS AVERAGE TOTAL ------------------- -------- -------- MAX STOR LOB VALUES 102.4K 512021

You can find the values within DB2 PM statistics record printout. They are also pointed out in miscellaneous. The values shown there are in megabytes as you can see in Example 7-3.

Example 7-3 DB2 PM statistics miscellaneous values

MISCELLANEOUS ------------------------------ BYPASS COL : 0.00 MAX SQL CASCAD LEVEL: 0.00


MAX STOR LOB VALUES : 1470.00

DB2 subsystem parameters for LOBsAs mentioned DB2 will create data spaces as needed. The LOB Manager will keep track of the amount used and manages the space within the data space. There are limits and they are specified in subsystem parameters in DSNZPARM. You will find them on the installation panel DSNTIP7 as shown in Example 7-4. They are defined as a maximum storage per user for LOB values in kilobytes (LOBVALA) and as a maximum storage per DB2 subsystem for LOB values in megabytes (LOBVALS). In DB2 V7 both ZPARMs can be changed without stopping DB2, refer to -SET SYSPARM in DB2 UDB for OS/390 and z/OS Version 7 Command Reference, SC26-9934, for details.

Figure 7-4 Panel DSNTIP7 showing storage LOB values

Table 7-1 Two subsystem parameters

� The value LOBVALA field establishes an upper limit for the amount of storage each user can have for storing LOB values. The value specified gives the numbers of kilobytes.

� The value LOBVALS field establishes an upper limit for the amount of storage per system that can be used for storing LOB values. The value specified gives the numbers of megabytes.

Because these definitions can have an impact on your operating system, specifically on paging and auxiliary storage, and depend on how heavy is the use of LOBs within your DB2 subsystem, the values of these settings should be chosen with care. Before setting the values of these new ZPARMs, you should consult your z/OS system programmer.

__________________________________________________________________________________ DSNTIP7 INSTALL DB2 - SIZES PANEL 2 ===> _ Check numbers and reenter to change: 1 USER LOB VALUE STORAGE ===> 2048 Max storage per user for LOB values in kilobytes 2 SYSTEM LOB VALUE STORAGE ===> 2048 Max storage per system for LOB values in megabytes 3 MAXIMUM LE TOKENS ===> 20 Maximum tokens at any time. 0-50 PRESS: ENTER to continue RETURN to exit HELP for more information __________________________________________________________________________________

Parameter Macro Panel Values

LOBVALA DSNSYSP DSNTIP7 1-2097152 KB

LOBVALS DSNSYSP DSNTIP7 1-51200 MB


If you run into a resource unavailable situation (SQLCODE -904) with data space pools, the reason codes you can expect are 00C900Dx (x = 1, 2 and 3). The reason codes mean respectively: user limit exceeded, system limit exceeded, and out of space condition in the data space.

7.2 Buffer pools and group buffer poolsThis section gives you an overview of buffer pool and group buffer pool considerations you have to be aware of when you are using large LOBs in production or non production environments.

Buffer poolsDepending upon how you have planned to use LOBs in your shop, you will most likely want to put LOB data in separate buffer pools, so as to not interfere with data used by online transactions or other work. If the LOBs have their own buffer pools, you have the capability of setting different thresholds than with your other buffer pools, without affecting other work. For example, you may want to set the virtual pool sequential steal threshold to 100%, taking full advantage of the sequential access, knowing that this buffer pool is only used for LOB pages. Also minimizing the buffer pool wide deferred write threshold (DWQT) almost to 0, this will force to asynchronously write buffers out of the buffer pool. At the data set level, the vertical deferred write threshold (VDWQT), will force write upon a data set level. It is expressed as a percentage of the virtual pool size for a single data set. This value should always be less than DWQT. In regards to dealing with LOB buffer pools, all three are worthwhile looking into. So again, it is not only meaningful but also a good advise to have your LOB buffer pools isolated from your other ‘normal’ buffer pools in your environment. See Figure 7-5.

Figure 7-5 Separation of LOB buffer pools

Separation of LOB Buffer Pools

BP32K1

BP22

BP20

BP3

Sequential

Buffer Pool

BP2

VPSIZESystem

BufferPool

Random

l

BufferPool

BP0

BP1

BP32K

BP12

Index

Buffer

Pool

BP10

VDWQT

DWQT

VPSEQT

S e

p a

r a t

e

DWQTVDWQT

Example of usual Buffer Pools LOB Buffer Pools

VPSEQT


Considerations for data sharingThe GBPCACHE SYSTEM option is allowed only for LOBs. For GBPCACHE SYSTEM page sets, the only pages that are written to the group buffer pool are LOB space map pages. All other data pages are written directly to disk, similar to GBPCACHE NONE page sets. SYSTEM is the default for LOB table spaces. See Figure 7-6.

Figure 7-6 LOB group buffer pool

As mentioned GBPCACHE SYSTEM is the default for a LOB table space and we recommended to use it. In a data sharing environment, if GBPCACHE CHANGED or GBPCACHE ALL is used instead for a LOG NO LOB table space, then a coupling facility failure could result in the table space being marked GRECP. Any kind of recovery of the table space will mark the LOB values as invalid and place the table space in the AUXW state. Again for LOB table spaces, choose GBPCACHE SYSTEM to avoid having large LOB values overwhelm the group buffer pool. Also, for LOB table spaces with the LOG NO attribute, GBPCACHE SYSTEM ensures that LOB values are written to DASD by commit time, thereby avoiding possible recovery problems caused by missing log data.

Storage estimate for caching LOB space mapsExample 7-4 shows the formula for estimating storage in the Coupling Facility for group buffer pools that cache LOB space map data (GBPCACHE SYSTEM).

Example 7-4 Calculation for LOB space maps

Data_entries = (U * D / 10) * R Data(MB) = Data_entries * P / 1024 Dir_entries = Data_entries + (U * (HP + VP)) Dir(MB) = 1.1 * Dir_entries * 0.2 / 1024 GBP(MB) = Data(MB) + Dir(MB)

G B P C A C H E S Y S TE MG B P C A C H E S Y S T E M fo r L O B fo r L O B

G R O U P B U F F E R P O O L G R O U P B U F F E R P O O L

s sn m D B M 1

B u ffe r P o o ls

s sn m D B M 1

B u ffe r P o o ls

C o u p lin g F a c ility C o u p lin g F ac ility

D B 2 d a ta sha ring env ironm e n t


RATIO = MIN(Dir_entries / Data_entries, 255)

Coming up with the formula, there are different values used and they have to be explained. Starting with the Data_entries = (U * D / 10) * R: The value U is the estimated degree of data sharing. The value D is based upon the number of data pages written to disk per second and the value of 1/10 is the estimate of LOB system pages that are written for every LOB data page. The value R is the average page residency time in the group buffer pool in seconds.

The Data(MB) = Data_entries * P / 1024: The value P is the page size (4, 8, 16, or 32) depending on the local buffer pool page size. At last the Dir_entries = Data_entries + (U * (HP + VP)): where HP The number of data pages defined for the hiperpool (the sum across all the members). VP The number of data pages defined for the virtual pool (the sum across all the data sharing members). Because the calculations are a little intricate, the following will step through a model. Assume that you have a two-member data sharing group for which you have determined the following information. See Example 7-5.

Example 7-5 Calculation of group buffer pool

The degree of data sharing is moderate (.7) There are 10 disk writes per second for across both members, peak rate The space map page is resident in the group buffer pool for 120 seconds The page size is 32 KB Member 1 is configured with a virtual pool of 20000 buffers and a hiperpool of 70000 buffers Member 2 is configured with a virtual pool of 10000 buffers and a hiperpool of 20000 buffers

The calculation is as follows: Data_entries = ((.7 * 10)/ 10) * 120 = 84Data(MB) = 84 * 32 / 1024 = 2.6 MBDir_entries = 84 + (.7 * (90000 + 30000)) = 84084Dir(MB) = 1.1 * 84084 * 0.2 / 1024 = 18.6 MBGBP(MB) = 2.6 MB + 18.6 MB = 21.2 MBRATIO = MIN (84084 / 84, 255) = 255

The above calculation indicates that the group buffer pool should be defined with an INITSIZE of 21.2 MB. The ratio is greater than the maximum value, which is not unusual with GBPCACHE(SYSTEM), so use the command ALTER GROUPBUFFERPOOL to change the ratio to 255.

7.3 LOBs performance considerationsIn this section we look at read and update performance, and provide recommendations.

The measurements shown were implemented with DB2 V6 in 1999 and were included in DB2 UDB for OS/390 Version 6 Performance Topics, SG24-5351. They are shown here because they still provide meaningful comparisons. More measurements are currently under way with DB2 V7 and more current disk technology, the results will be included as soon as available.

Note about the value D: Do not use the number of pages written to the group buffer pool as D. Instead it must be a count of distinct pages. This value can be determined by using field QBSTPWS from IFCID 0002. You can find it using the DB2 PM and looking into the statistics report (PAGES WRITTEN field). This will give you the right value.


7.3.1 Logging with LOBsTo reduce the volume of logging, you can specify LOG NO in your CREATE LOB TABLESPACE statement. This suppresses writing redo records.

There are no UNDO records for LOB updates (except for system pages, space map) even with LOG YES. LOBs always insert the new value at a different place and delete the old one at commit, marking the old space as free.

To give you an idea of the amount of data being written to the log, look at Example 7-6 which shows the DSN1LOGP output for a LOB insert of a using LOG YES.

Example 7-6 Inserting a 1 MB LOB with LOG YES

URID(0002AB431314) LRSN(B76312AC4AE6) DBID(0130) OBID(000D) PAGE(00000001) TYPE( REDO ) SUBTYPE(UPDATE SPACE MAP) CLR(NO) PROCNAME(DSNISGSU)

URID(0002AB431314) LRSN(B76312AC4AE7) DBID(0130) OBID(000D) PAGE(00000001) TYPE( REDO ) SUBTYPE(CURRENT LAST ENTRY IN SPACE MAP PAGE) CLR(NO)PROCNAME(DSNISGSU) URID(0002AB431314) LRSN(B76312AC4AE7) DBID(0130)OBID(000D) PAGE(00000002) TYPE( REDO ) SUBTYPE(FORMAT PAGE OR MODIFY SPACE MAP) CLR(NO) PROCNAME(DSNISGRT) URID(0002AB431314) LRSN(B76312AC4AE7) DBID(0130)OBID(000D) PAGE(00000002) TYPE( UNDO REDO ) SUBTYPE(INSERT IN A DATA PAGE) CLR(NO) PROCNAME(DSNISGRT) URID(0002AB431314) LRSN(B76312AC4AE8) DBID(0130) OBID(000D) PAGE(00000001) TYPE( UNDO REDO ) SUBTYPE(UPDATE SPACE MAP) CLR(NO) PROCNAME(DSNISGSU) LRSN(B76312AC4D71) DBID(0130) OBID(0010) TYPE(PAGE SET CONTROL) SUBTYPE(PAGE SET OPEN)

URID(0002AB431314) LRSN(B76312AC4EF2) DBID(0130) OBID(0010) PAGE(00000005) TYPE( UNDO REDO ) SUBTYPE(ALLOC/DEALLOC OF SPACE IN LOB SPACE MAP PAGE)CLR(NO) PROCNAME(DSNOALLO) URID(0002AB431314) LRSN(B76312AC4EF3) DBID(0130) OBID(0010) PAGE(00000001) TYPE( UNDO REDO ) SUBTYPE(LOB HIGH LEVEL SPACE MAP PAGE UPDATE) CLR(NO)PROCNAME(DSNOALLO) URID(0002AB431314) LRSN(B76312AC4EF5) DBID(0130) OBID(0010) PAGE(00000005) TYPE( UNDO REDO ) SUBTYPE(ALLOC/DEALLOC OF SPACE IN LOB SPACE MAP PAGE)CLR(NO) PROCNAME(DSNOALLO) URID(0002AB431314) LRSN(B76312AC4EFC) DBID(0130) OBID(0010) PAGE(00000015) TYPE( REDO ) SUBTYPE(LOB MAP CHANGE) CLR(NO) PROCNAME(DSNOFLMP) URID(0002AB431314) LRSN(B76312AC4EFD) DBID(0130)


OBID(0010) PAGE(00000015) TYPE( REDO ) SUBTYPE(LOB DATA PAGE CHANGE) CLR(NO) PROCNAME(DSNOLINS)

URID(0002AB431314) LRSN(B76312AC4F0B) DBID(0130)OBID(0010) PAGE(00000016) TYPE( REDO ) SUBTYPE(LOB DATA PAGE CHANGE) CLR(NO) PROCNAME(DSNOLINS)

[ LOB DATA PAGE CHANGE REPEATS n TIMES ]

URID(0002AB431314) LRSN(B76312ACD1FE) DBID(0130)OBID(0014) PAGE(00000003) TYPE( UNDO REDO ) SUBTYPE(TYPE 2 INDEX UPDATE) CLR(NO) PROCNAME(DSNKINSL)

LRSN(B76312D1B66E) DBID(0130) OBID(0014) TYPE(PAGE SET CONTROL) SUBTYPE(PAGE SET WRITE)

The log entries marked as bold text, occurring n times where n depends on the size of your LOB value, represent the additional overhead for specifying LOG YES for the LOB table space. Every LOB DATA PAGE CHANGE log entry contains data of a single LOB data page with those values you insert into your LOB column. You have as many LOB DATA PAGE CHANGE log entries in your log as needed for storing your LOB value.

You will not find the bold text log entries if you have created your LOB table space using LOG NO. This means that only system pages for the LOB value are written to the log, no LOB data is written to the log.

The amount of data to be logged can grow rapidly when you use the LOG YES clause.

7.3.2 LOBs processingThe maximum size of a LOB column is 2 GB. You can also use LOBs to store data, for example, long characters, which do not fit entirely within DB2’s largest page size of 32 KB.

Although you can manipulate LOB columns like any other data type, there are a number of issues:

� They are subject to the same restrictions as long strings. Substrings of LOBs which are less than 256 bytes can be CAST as CHAR to avoid these restrictions.

� Acquiring buffers in a program to accommodate a large LOB can be difficult.

Since LOBs are essentially long strings, you can use string functions to manipulate them and parts of them. For example, SUBSTR(LOB,1,200) will retrieve the first 200 bytes of the LOB. This can more easily be managed in an application program.

LOBs are inserted, selected, deleted, and updated using standard SQL, although there are some special considerations:

� A LOB locator, which is a 4-byte value stored in a host variable, can be used to reference the LOB, and can be used wherever you would use a LOB. The locator essentially acts as a pointer to the LOB. For full details, refer to DB2 UDB for OS/390 and z/OS Version 7 Application Programming and SQL Guide, SC26-9933.

� The LOB update operation does not update in place, it performs a LOB delete followed by an insert.

� The LOB delete is a logical delete: it flags the space as available.


7.3.3 LOBs read performanceWe have measured the efficiency of reading data from tables that contain LOB data. We created four different tables. The first was the reference case and consisted of a non-LOB 20 KB character column. The table space page size had to be 32 KB. Operations performed on this table provided the baseline against which we could compare the efficiency of LOB processing. See Figure 7-7.

Figure 7-7 Read performance

The 20 KB LOB table was conceptually identical to the first one except that the 20 KB character column was defined as a CLOB. We created a LOB table space and auxiliary table and index to support this. For our LOB table spaces, we used a 32 KB page size, so that comparisons with the non-LOB table were reasonable.

The other two tables were defined with 200 KB and 2 MB LOBs, respectively.

When we retrieved LOB data into the host application, we repeatedly called SUBSTR to “walk” down the entire length of the column. We used the same technique to process LOB data with and without the use of LOB locators. Any differences in the performance of processing LOBs can, therefore, be attributed solely to the use of locators or the processing of LOBs into host variables.

The diagram above shows the elapsed and CPU time taken to process non-LOB and LOB data of different lengths and shows the following results:

� LOB processing is more efficient for larger LOB columns.

The time taken to process a select of 20 KB worth of LOB data into a host variable is 0.027 seconds. In other words DB2 can process 740.7 KB LOB data per second for a 20 KB LOB column. To select a 2 MB LOB into a host variable (that is, 100 times more data) does not take 100 times longer; rather, DB2 can process 2306.8 KB LOB data per second for a 2 MB LOB column.

LOBs read performance

non-LOB 20 KB 20 KB 200 KB 2 MBSelect into :HV 0.025

(0.0006)0.027

(0.0008)0.117

(0.0023)0.867

(0.0175) KB/sec 800.0 740.7 1,709.4 2,306.8

Select into locator 0.020(0.0007)

0.029(0.0007)

0.046(0.0007)

All measurements in secondsElapsed time is on the top line, CPU time shown in bracketsProcessor is LPAR of RX5, disks are RAMAC 2

Large LOBs can be processed (KB/sec) more efficiently than small

For 2 MB LOBs, locators offer superior performance and lower CPU overhead:

nearly 19 times faster elapsed25 time less CPU required

We recommend you use LOB locators


� The efficiency by which large LOBs can be processed is increased enormously by using LOB locators.

With 20 KB of LOB data, there is only a small improvement in performance using LOB locators. At 2 MB we saw that it took nearly 19 times longer and used 25 times more CPU to select a LOB into a host variable rather than into a LOB locator.

� You get very consistent responses and CPU times reading LOB data using locator variables regardless of the size of the LOB column.

These measurements lead us to recommend that you use LOB locators when selecting LOB data.

7.3.4 Comparing SQL accounting profilesThere are good reasons for the use of LOBs, but there are also good reasons to consider using VARCHAR instead, if possible. Even when dealing with the wide variety of additional data types, such as audio, video or mixed text, if they are below 32 KB in size, you should consider the possibility of storing them into a VARCHAR column. The main reasons for still wanting to use VARCHAR or VARGRAPHIC, for your small (<32KB) multimedia objects, are that there is added complexity with LOBs, and that there are more SQL calls executed to process a LOB than when processing a VARCHAR column. As an example, we have setup an environment with a simple base table and its associated LOB. The LOB is 32 KB in size. On the other side we have a normal table that contains the same structure as the base table, except that it has a VARCHAR column. The VARCHAR column can also have 32 KB. For our comparison we now use two different batch application programs, both using host variables. One selects the LOB and the other selects the VARCHAR column into a host variable, as shown in Example 7-7.

Example 7-7 Select into host variable

EXEC SQL SELECT DOCUMENT INTO :LO-DOCUMENT FROM PAOLO.CLOB_BASE_TABLE WHERE DOCUMENT_NR = :LO-DOCUMENT-NREND-EXEC

We now look at the accounting trace output provided by DB2 PM and reported in Figure 7-8. The values shown for DB2 Class 2 elapsed time clearly indicates which strategy to use for your small (<32 KB) objects.

Example 7-8 Accounting trace output

SELECTING A LOB AVERAGE APPL(CL.1) DB2 (CL.2) ----------- ---------- ---------- ELAPSED TIME 0.803155 0.557689 CPU TIME 0.033644 0.012224 _____________________________________

SELECTING A VARCHARAVERAGE APPL(CL.1) DB2 (CL.2) ------------ ---------- ---------- ELAPSED TIME 0.536118 0.284875 CPU TIME 0.061166 0.040178


7.3.5 LOBs write performanceWe also measured the performance of update, delete, and insert operations against LOB data. The scenario we used was the same as for LOB reads: one non-LOB table with a 20 KB CHAR as a control for the 20 KB LOB table, and tables with LOB sizes of 200 KB and 2 MB. The processor in an RX5 LPAR and LOG NO was specified for the LOB runs.

We examined the cost of an insert operation using a host variable compared with using a LOB locator with different sized LOBs. We also measured the cost of a delete and an update. See Figure 7-8.

Figure 7-8 Write performance

LOB insertIt is more expensive to process 20 KB of LOB data, even if a LOB locator is used, than processing an equivalent volume of non-LOB data. As with select processing, the greater the LOB length, the greater the efficiency of insert processing — to insert 100 times the data does not require 100 times the resource. The relative improvement of insert efficiency with respect to LOB size results, however, are much less dramatic than for select processing. The reason is that the cost of preformatting disk space to accommodate the newly inserted LOB increases with the size of the LOB and it accounts for most of the elapsed time.

There is relatively little difference in insert performance if a LOB locator is used.

LOB updateInternally a LOB update is equivalent to a LOB insert and a LOB delete. Most of the cost of a LOB update can, therefore, be attributed to the cost of the LOB insert.

LOBs write performance

2 MB insert takes about 2 secondsinsert more efficient as length of LOB data increases

Delete is very quick - logical delete

Update is roughly the same as an insertinternally consists of insert+delete so this makes sense

NON-LOB 20 KB 20 KB 200 KB 2 MB

Insert with :hv 0.011(0.0007)

0.025(0.0012)

0.219(0.0033)

2.029(0.0194)

Insert with locator 0.028(0.0012)

0.187(0.0028)

2.08(0.0199)

Update (see notes) 0.0036(0.0008)

0.027(0.0013)

0.223(0.0033)

2.2290.021)

Delete 0.03(0.001)

0.024(0.001)

0.035(0.001)

0.045(0.001)

All measurements in secondsElapsed time is on the top line, CPU time shown in brackets


LOB deleteA LOB delete is a logical delete, and so is relatively cheap and gives consistently good performance. The CPU cost was independent of the size of the LOB column (to 2MB). The larger the LOB column, the greater the number of space map pages that have to be updated to reflect the logically deleted row. There is, therefore, a relationship between elapsed time and LOB size for deletes. The larger the LOB column, the greater the efficiency of the delete operation. DB2 can delete 833 KB per second if the LOB is 20 KB in length. With a 2 MB LOB, DB2 can delete 45,511 KB per second — more than 50 times faster.

7.3.6 IFCIDs enhancements for LOBs If you have defined LOBs within your system, the need of tracking their performance and watching over all resources used by your application retrieving LOB data, can become very important to you. The use of LOBs may increase in the near future and getting to know some main counters within the IFCID environment can help you there. Because the IFCIDs are part of your DB2 traces, you may have to start them. By chance you already have them active in your DB2 system. There is a collection of the current IFCIDs providing information regarding LOBs. See Table 7-2.

Table 7-2 Current IFCID providing information regarding LOBs

IFCID Fields Description

IFCID 2 DB2 statistics record

QXSTLOBV Maximum storage used for LOB values

QXCRATB Number of CREATE AUXILIARY TABLE statements

QXHLDLOC Number of HOLD LOCATOR statements

QXFRELOC Number of FREE LOCATOR statements

IFCID 3 DB2 accounting record

QXSTLOBV Maximum storage used for LOB values

QWACLRN Number of log records written

QWACLRAB Total number of bytes of log records written

IFCID 18 Ends sequential scan, index scan or insert. The additional pages scanned in a LOB table space and for a count of the number of LOBs updated. Other fields in IFCID 18 are only applicable to the base table.

QW0018PL Additional pages scanned in a LOB table space

QW0018UL Count of LOB data pages updated (either by an SQL INSERT or an SQL UPDATE)

IFCID 20 Summary of page, row and LOB locks held and lock escalation.

QW0020TP Maximum number of page, row and LOB locks held

QW0020PL Maximum number of either page, row or LOB locks held for the thread

QW0020F5 LOB table space

QW0020R3 LOB lock


IFCID 21 Detail lock trace LOB lock type.

QW0021ML LOB lock type (value '30 'x)

QW0021KX ID of resource for LOB locks

QW0021K6 Row ID

QW0021K7 Version number

IFCID 23 Record utility start information

QW0023PH CHECKLOB is a new phase for the CHECK LOB utility

IFCID 24 Records utility object or phase change record


IFCID 25 Record utility start information


IFCID 44 Records lock suspension. LOB lock type record utility start information



QW0044K6 Row ID


IFCID 58 End of SQL statement execution. Add fields for the additional pages scanned in a LOB table space and for a count of the number of LOBs updated. Other fields in IFCID 58 are only applicable to the base table.

QW0058PL Additional pages scanned in a LOB table space

QW0058UL Count of LOB data pages updated (either by an SQL INSERT oran SQL UPDATE)

IFCID 62 DDL (and other) execution statement start

QW0062CX Create auxiliary table. Value x'F2'

QW0062HL Hold locator. Value x'CE'.

QW0062FL Free locator. Value x'CF'

IFCID 105 Maps the internal DBID and OBID to the database and table space name. also maps the DBID and OBID of the LOB table space and index on the auxiliary table to the LOB table space name and name of the index on the auxiliary table. DDL (and other) execution statement start.

IFCID 106 System Initialization Parameters

QWP1LVA Bytes for LOB values - per user

QWP1LVS Bytes for LOB values - per system

IFCID 107 Open/close information. Also records open and close for LOB table spaces and indexes on auxiliary tables

IFCID 141 Records grants and revokes.



Traces to use with LOBs� IFCID 58

– QW0058PL - pages scanned in LOB table space– QW0058UL - pages updated in LOB table spaces

� IFCID 321 and 322

– 321 - begin force-at-commit– 322 - end force-at-commit– belong to the same classes as other I/O wait events:

• Accounting classes 3 and 8• Monitor classes 3 and 8• Performance class 4

QW0140BS For a LOB table space

QW0140BT For an auxiliary table

IFCID 150 Lock information for a given agent.



QW0150K6 Row ID


IFCID 172 Deadlock trace.

QW0172MO LOB lock type (value '30 'x)


QW0172K6 Row ID


IFCID 185 Data capture information deadlock trace.

QW0185ST For a LOB column, this is the data type of the LOB

QW0185LE For a LOB column, this is the length of the indicator column

IFCID 196 Time-out trace. Lock types are not explicitly specified for this trace

QW0196FR This value represents a LOB lock '30 'X


QW0196K6 Row ID


IFCID 306 Records log records. Note that when LOG NO has been specified for a LOB table space, log records will not be written for the value of the LOB.

IFCID 321 Trace record to trace the beginning of a force-at-commit.

IFCID 322 Trace record to trace the end of a force-at-commit.

QW0322NP Number of pages written



� QXSTLOBV

– Maximum storage used for LOBs in data spaces– Value is in kilobytes for accounting– Value is in megabytes for statistics

7.3.7 LOBs recommendations� Use LOBS for what they are intended — large (>32 KB) objects.

On balance, we observed that the larger the LOB, the greater the efficiency. LOB insert (and therefore update) processing is expensive for small length LOB columns. We recommend, therefore, that you use LOBs only for the purpose for which they were intended — processing large objects over 32 KB in size. You must take into account that there is only 1 LOB per page; if you have specified a pagesize of 32 KB, and a row of 100 bytes, the overhead in dealing with each row in terms of I/O and CPU cannot be negligible.

� Use the correct page size

For really large LOBs, page size must be 32 KB. Since DB2 places only one LOB per page, space can be wasted for small LOBs (less than the page size). Use 4 KB page for LOB tablespace to minimize wasted disk space and potentially much higher I/O time if both small and large LOBs exist, as it is found to be a very common situation.

� Use locators to manipulate LOBs.

LOB locators process LOB data consistently and more efficiently than using host variables, and avoid application buffering problems.

� Do not use LOBS as a means of normalizing data.

We do not recommend that you use LOBs as a technique to obtain performance improvements that normalization and proper physical design can achieve. Note that:

– Any improvements are entirely application dependent.

– At the physical design stage, you should have been able to identify the columns that are infrequently accessed and the relationship across the columns. Depending on the application, there may be a net benefit in placing some columns into its own base table, keyed on employee number and holding the 20 KB VARCHAR and possibly other infrequently accessed columns.

– Do not use LOBs as a means to deactivate REDO logging.

The additional cost, particularly when processing small LOBs, means you are likely to create a different bottleneck from the perceived one of logging that you were trying to solve. Use LOBs only for their intended purpose.

� Expect overhead when running LOAD

You should be aware that the LOAD utility processes LOBs using insert mode processing. To estimate very roughly the elapsed time for LOAD, assume that the cost to load each row is equivalent to the cost of randomly inserting every row.

� Tune VSAM and GRS with data sharing

In data sharing environments, we observed some contention during drop LOB table space and unusually high Coupling Facility (CF) CPU activity. We could attribute this to VSAM making use of GRS. When we specified GRS=STAR, these problems were eliminated. We recommend that you investigate the GRS setting in data sharing environments if you are observing high CF CPU activity.


Appendix A. Sample jobs output

In this appendix we provide the output of sample jobs used to illustrate the examples included in this redbook. It is structured as follows:

� DDL for a LOB environment� Sample of DB2 PM accounting trace output� Using file reference variables

A


A.1 DDL for a LOB environmentThis sample is the output of the DDL that creates the environment for our sample programs.

DSNTIAD - SAMPLE DYNAMIC SQL PROGRAM 2.0

CREATE TABLESPACE CLOBBTS IN PAOLO USING STOGROUP PAOLOSG1 PRIQTY 1000 SECQTY 100 ERASE NO BUFFERPOOL BP12 COMPRESS YES SEGSIZE 64 LOCKSIZE ANY CLOSE NO DSNT400I SQLCODE = 000, SUCCESSFUL EXECUTION CREATE TABLE PAOLO.CLOB_BASE_TABLE ( DOCUMENT_NR CHAR(10) NOT NULL WITH DEFAULT , DESCRIPTION CHAR(32) NOT NULL WITH DEFAULT

, ROW_ID ROWID NOT NULL GENERATED ALWAYS , DOCUMENT CLOB(1G) NOT NULL WITH DEFAULT ) IN PAOLO.CLOBBTS DSNT400I SQLCODE = 000, SUCCESSFUL EXECUTION COMMIT DSNT400I SQLCODE = 000, SUCCESSFUL EXECUTION CREATE LOB TABLESPACE CLOBATS IN PAOLO USING STOGROUP PAOLOSG1 PRIQTY 50000 SECQTY 50000 ERASE NO BUFFERPOOL BP32K LOCKSIZE LOB

CLOSE NO LOG NO DSNT400I SQLCODE = 000, SUCCESSFUL EXECUTION CREATE AUXILIARY TABLE PAOLO.CLOB_AUX_TABLE IN PAOLO.CLOBATS STORES PAOLO.CLOB_BASE_TABLE COLUMN DOCUMENT DSNT400I SQLCODE = 000, SUCCESSFUL EXECUTION COMMIT DSNT400I SQLCODE = 000, SUCCESSFUL EXECUTION CREATE TYPE 2 UNIQUE INDEX PAOLO.CLOBAIX ON PAOLO.CLOB_AUX_TABLE USING STOGROUP PAOLOSG1 PRIQTY 100 SECQTY 10

FREEPAGE 10 PCTFREE 10 BUFFERPOOL BP11 CLOSE NO


DSNT400I SQLCODE = 000, SUCCESSFUL EXECUTION COMMIT DSNT400I SQLCODE = 000, SUCCESSFUL EXECUTION

A.2 Sample of DB2 PM accounting trace outputWithin this DB2 PM accounting output you will find special counters related to the LOB processing and performance. Look at the miscellaneous part of the output, you will find some highlighted counters that may be of interest to you when analyzing LOBs performance.

1 LOCATION: DB2G DB2 PERFORMANCE MONITOR (V7) PAGE: 2-14 GROUP: N/P ACCOUNTING REPORT - LONG REQUESTED FROM: NOT SPECIFIED MEMBER: N/P TO: NOT SPECIFIED SUBSYSTEM: DB2G ORDER: PLANNAME INTERVAL FROM: 03/18/02 22:36:2 DB2 VERSION: V7 SCOPE: MEMBER TO: 03/18/02 23:19:3

PLANNAME: SAMPLE3

ELAPSED TIME DISTRIBUTION CLASS 2 TIME DISTRIBUTION ---------------------------------------------------------------- ------------------------------------------------------------ APPL !===========================================> 86% CPU !> 1% DB2 !==> 4% NOTACC !==============> 28% SUSP !=====> 10% SUSP !===================================> 71%

AVERAGE APPL(CL.1) DB2 (CL.2) IFI (CL.5) CLASS 3 SUSPENSIONS AVERAGE TIME AV.EVENT HIGHLIGHTS ------------ ---------- ---------- ---------- -------------------- ------------ -------- -------------------------- ELAPSED TIME 24:30.4256 3:29.97658 N/P LOCK/LATCH(DB2+IRLM) 1:28.359418 55.80 #OCCURRENCES : 5 NONNESTED 24:30.4256 3:29.97658 N/A SYNCHRON. I/O 3.145753 71.20 #ALLIEDS : 5 STORED PROC 0.000000 0.000000 N/A DATABASE I/O 3.145753 71.20 #ALLIEDS DISTRIB: 0 UDF 0.000000 0.000000 N/A LOG WRITE I/O 0.000000 0.00 #DBATS : 0 TRIGGER 0.000000 0.000000 N/A OTHER READ I/O 0.000000 0.00 #DBATS DISTRIB. : 0 OTHER WRTE I/O 0.000000 0.00 #NO PROGRAM DATA: 0 CPU TIME 4.398020 1.660921 N/P SER.TASK SWTCH 58.006180 15.20 #NORMAL TERMINAT: 0 AGENT 4.398020 1.660921 N/A UPDATE COMMIT 40.082702 1.00 #ABNORMAL TERMIN: 5 NONNESTED 4.398020 1.660921 N/P OPEN/CLOSE 0.078285 0.20 #CP/X PARALLEL. : 0 STORED PRC 0.000000 0.000000 N/A SYSLGRNG REC 0.021327 0.20 #IO PARALLELISM : 0 UDF 0.000000 0.000000 N/A EXT/DEL/DEF 17.741402 13.20 #INCREMENT. BIND: 0 TRIGGER 0.000000 0.000000 N/A OTHER SERVICE 0.082464 0.60 #COMMITS : 0 PAR.TASKS 0.000000 0.000000 N/A ARC.LOG(QUIES) 0.000000 0.00 #ROLLBACKS : 5 ARC.LOG READ 0.000000 0.00 #SVPT REQUESTS : 0 SUSPEND TIME N/A 2:29.51135 N/A STOR.PRC SCHED 0.000000 0.00 #SVPT RELEASE : 0 AGENT N/A 2:29.51135 N/A UDF SCHEDULE 0.000000 0.00 #SVPT ROLLBACK : 0 PAR.TASKS N/A 0.000000 N/A DRAIN LOCK 0.000000 0.00 MAX SQL CASC LVL: 0 CLAIM RELEASE 0.000000 0.00 UPDATE/COMMIT : 0.20 NOT ACCOUNT. N/A 58.804310 N/A PAGE LATCH 0.000000 0.00 SYNCH I/O AVG. : 0.044182 DB2 ENT/EXIT N/A 101.20 N/A NOTIFY MSGS 0.000000 0.00 EN/EX-STPROC N/A 0.00 N/A GLOBAL CONTENTION 0.000000 0.00 EN/EX-UDF N/A 0.00 N/A COMMIT PH1 WRITE I/O 0.000000 0.00 DCAPT.DESCR. N/A N/A N/P ASYNCH IXL REQUESTS 0.000000 0.00 LOG EXTRACT. N/A N/A N/P TOTAL CLASS 3 2:29.511350 142.20

GLOBAL CONTENTION L-LOCKS AVERAGE TIME AV.EVENT GLOBAL CONTENTION P-LOCKS AVERAGE TIME AV.EVENT ------------------------------------- ------------ -------- ------------------------------------- ------------ -------- L-LOCKS 0.000000 0.00 P-LOCKS 0.000000 0.00 PARENT (DB,TS,TAB,PART) 0.000000 0.00 PAGESET/PARTITION 0.000000 0.00 CHILD (PAGE,ROW) 0.000000 0.00 PAGE 0.000000 0.00 OTHER 0.000000 0.00 OTHER 0.000000 0.00


PLANNAME: SAMPLE3

SQL DML AVERAGE TOTAL SQL DCL TOTAL SQL DDL CREATE DROP ALTER LOCKING AVERAGE -------- -------- -------- -------------- -------- ---------- ------ ------ ------ ---------------------- -------- ---- SELECT 0.00 0 LOCK TABLE 0 TABLE 0 0 0 TIMEOUTS 0.00 INSERT 0.20 1 GRANT 0 CRT TTABLE 0 N/A N/A DEADLOCKS 0.00 UPDATE 0.00 0 REVOKE 0 DCL TTABLE 0 N/A N/A ESCAL.(SHARED) 0.00 DELETE 0.00 0 SET CURR.SQLID 0 AUX TABLE 0 N/A N/A ESCAL.(EXCLUS) 0.00 SET HOST VAR. 247 INDEX 0 0 0 MAX PG/ROW LOCKS HELD 0.60 DESCRIBE 0.00 0 SET CUR.DEGREE 0 TABLESPACE 0 0 0 LOCK REQUEST 87.60

Appendix A. Sample jobs output 139

DESC.TBL 0.00 0 SET RULES 0 DATABASE 0 0 0 UNLOCK REQUEST 86.80 PREPARE 0.00 0 SET CURR.PATH 0 STOGROUP 0 0 0 QUERY REQUEST 0.00 OPEN 0.00 0 SET CURR.PREC. 0 SYNONYM 0 0 N/A CHANGE REQUEST 0.00 FETCH 0.00 0 CONNECT TYPE 1 0 VIEW 0 0 N/A OTHER REQUEST 0.00 CLOSE 0.00 0 CONNECT TYPE 2 0 ALIAS 0 0 N/A LOCK SUSPENSIONS 54.80 SET CONNECTION 0 PACKAGE N/A 0 N/A IRLM LATCH SUSPENSIONS 0.00 RELEASE 0 PROCEDURE 0 0 0 OTHER SUSPENSIONS 0.00 DML-ALL 0.20 1 CALL 0 FUNCTION 0 0 0 TOTAL SUSPENSIONS 54.80 ASSOC LOCATORS 0 TRIGGER 0 0 N/A ALLOC CURSOR 0 DIST TYPE 0 0 N/A HOLD LOCATOR 0 FREE LOCATOR 0 TOTAL 0 0 0 DCL-ALL 247 RENAME TBL 0 COMMENT ON 0 LABEL ON 0

NORMAL TERM. AVERAGE TOTAL ABNORMAL TERM. TOTAL IN DOUBT TOTAL DRAIN/CLAIM AVERAGE --------------- -------- -------- ----------------- -------- -------------- -------- -------------- -------- ---- NEW USER 0.00 0 APPL.PROGR. ABEND 5 APPL.PGM ABEND 0 DRAIN REQUESTS 0.00 DEALLOCATION 0.00 0 END OF MEMORY 0 END OF MEMORY 0 DRAIN FAILED 0.00 APPL.PROGR. END 0.00 0 RESOL.IN DOUBT 0 END OF TASK 0 CLAIM REQUESTS 1.60 RESIGNON 0.00 0 CANCEL FORCE 0 CANCEL FORCE 0 CLAIM FAILED 0.00 DBAT INACTIVE 0.00 0 RRS COMMIT 0.00 0

DATA CAPTURE AVERAGE TOTAL DATA SHARING AVERAGE TOTAL QUERY PARALLELISM AVERAGE ----------------- -------- -------- ------------------- -------- -------- ---------------------------- -------- ---- IFI CALLS MADE N/P N/P GLOBAL CONT RATE(%) N/C N/A MAXIMUM MEMBERS USED N/A RECORDS CAPTURED N/P N/P FALSE CONT RATE(%) N/C N/A MAXIMUM DEGREE N/A LOG RECORDS READ N/P N/P L-LOCKS XES RATE(%) N/C N/A GROUPS EXECUTED 0.00 ROWS RETURNED N/P N/P LOCK REQ - PLOCKS 0.00 0 RAN AS PLANNED 0.00 RECORDS RETURNED N/P N/P UNLOCK REQ - PLOCKS 0.00 0 RAN REDUCED 0.00 DATA DESC. RETURN N/P N/P CHANGE REQ - PLOCKS 0.00 0 ONE DB2-COORDINATOR = NO 0.00 TABLES RETURNED N/P N/P LOCK REQ - XES 0.00 0 ONE DB2-ISOLATION LEVEL 0.00 DESCRIBES N/P N/P UNLOCK REQ - XES 0.00 0 ONE DB2-DCL TEMPORARY TABLE 0.00 CHANGE REQ - XES 0.00 0 SEQUENTIAL-CURSOR 0.00 SUSPENDS - IRLM 0.00 0 SEQUENTIAL-NO ESA SORT 0.00 SUSPENDS - XES 0.00 0 SEQUENTIAL-NO BUFFER 0.00 SUSPENDS - FALSE 0.00 0 SEQUENTIAL-ENCLAVE SERVICES 0.00 INCOMPATIBLE LOCKS 0.00 0 MEMBER SKIPPED (%) N/C NOTIFY MSGS SENT 0.00 0 DISABLED BY RLF 0.00 REFORM PARAL-CONFIG 0.00 REFORM PARAL-NO BUF 0.00


PLANNAME: SAMPLE3

STORED PROCEDURES AVERAGE TOTAL UDF AVERAGE TOTAL TRIGGERS AVERAGE TOTAL ----------------- -------- -------- --------- -------- -------- ----------------- -------- -------- CALL STATEMENTS 0.00 0 EXECUTED 0.00 0 STATEMENT TRIGGER 0.00 0 ABENDED 0.00 0 ABENDED 0.00 0 ROW TRIGGER 0.00 0 TIMED OUT 0.00 0 TIMED OUT 0.00 0 SQL ERROR OCCUR 0.00 0 REJECTED 0.00 0 REJECTED 0.00 0

LOGGING AVERAGE TOTAL ROWID AVERAGE TOTAL RID LIST AVERAGE TOTAL ------------------- -------- -------- ------------- -------- -------- ------------------- -------- -------- LOG RECORDS WRITTEN 12.60 63 DIRECT ACCESS 0.00 0 USED 0.00 0 TOT BYTES WRITTEN 7051.60 35258 INDEX USED 0.00 0 FAIL-NO STORAGE 0.00 0 TS SCAN USED 0.00 0 FAIL-LIMIT EXCEEDED 0.00 0

AVERAGE SU CLASS 1 CLASS 2 DYNAMIC SQL STMT AVERAGE TOTAL MISCELLANEOUS AVERAGE TOTAL ------------ -------------- -------------- ------------------ -------- -------- ------------------- -------- ------- CPU 50588.20 19104.60 REOPTIMIZATION 0.00 0 MAX STOR LOB VALUES 102.4K 512021 AGENT 50588.20 19104.60 NOT FOUND IN CACHE 0.00 0 NONNESTED 50588.20 19104.60 FOUND IN CACHE 0.00 0 STORED PRC 0.00 0.00 IMPLICIT PREPARES 0.00 0 UDF 0.00 0.00 PREPARES AVOIDED 0.00 0 TRIGGER 0.00 0.00 STMT INVALID (MAX) 0.00 0 PAR.TASKS 0.00 0.00 STMT INVALID (DDL) 0.00 0

BP0 AVERAGE TOTAL BP11 AVERAGE TOTAL BP12 AVERAGE --------------------- -------- -------- --------------------- -------- -------- --------------------- -------- ----- BPOOL HIT RATIO (%) 98.56 N/A BPOOL HIT RATIO (%) 42.86 N/A BPOOL HIT RATIO (%) 100.00 HPOOL HIT RATIO (%) 0.00 N/A HPOOL HIT RATIO (%) 0.00 N/A HPOOL HIT RATIO (%) N/C GETPAGES 41.80 209 GETPAGES 1.40 7 GETPAGES 0.80 GETPAGES-FAILED 0.00 0 GETPAGES-FAILED 0.00 0 GETPAGES-FAILED 0.00 BUFFER UPDATES 0.00 0 BUFFER UPDATES 0.60 3 BUFFER UPDATES 0.80 SYNCHRONOUS WRITE 0.00 0 SYNCHRONOUS WRITE 0.20 1 SYNCHRONOUS WRITE 0.00


SYNCHRONOUS READ 0.00 0 SYNCHRONOUS READ 0.80 4 SYNCHRONOUS READ 0.00 SEQ. PREFETCH REQS 0.00 0 SEQ. PREFETCH REQS 0.00 0 SEQ. PREFETCH REQS 0.00 LIST PREFETCH REQS 0.00 0 LIST PREFETCH REQS 0.00 0 LIST PREFETCH REQS 0.00 DYN. PREFETCH REQS 0.20 1 DYN. PREFETCH REQS 0.00 0 DYN. PREFETCH REQS 0.00 PAGES READ ASYNCHR. 0.60 3 PAGES READ ASYNCHR. 0.00 0 PAGES READ ASYNCHR. 0.00 HPOOL WRITES 0.00 0 HPOOL WRITES 0.00 0 HPOOL WRITES 0.00 HPOOL WRITES-FAILED 0.00 0 HPOOL WRITES-FAILED 0.00 0 HPOOL WRITES-FAILED 0.00 PAGES READ ASYN-HPOOL 0.00 0 PAGES READ ASYN-HPOOL 0.00 0 PAGES READ ASYN-HPOOL 0.00 HPOOL READS 0.00 0 HPOOL READS 0.00 0 HPOOL READS 0.00 HPOOL READS-FAILED 0.00 0 HPOOL READS-FAILED 0.00 0 HPOOL READS-FAILED 0.00


PLANNAME: SAMPLE3

TOT4K AVERAGE TOTAL BP32K AVERAGE TOTAL TOTAL AVERAGE TO --------------------- -------- -------- --------------------- -------- -------- --------------------- -------- ----- BPOOL HIT RATIO (%) 96.82 N/A BPOOL HIT RATIO (%) 97.79 N/A BPOOL HIT RATIO (%) 97.77 HPOOL HIT RATIO (%) 0.00 N/A HPOOL HIT RATIO (%) 0.00 N/A HPOOL HIT RATIO (%) 0.00 GETPAGES 44.00 220 GETPAGES 3278.00 16390 GETPAGES 3322.00 16 GETPAGES-FAILED 0.00 0 GETPAGES-FAILED 0.00 0 GETPAGES-FAILED 0.00 BUFFER UPDATES 1.40 7 BUFFER UPDATES 3210.20 16051 BUFFER UPDATES 3211.60 16 SYNCHRONOUS WRITE 0.20 1 SYNCHRONOUS WRITE 0.00 0 SYNCHRONOUS WRITE 0.20 SYNCHRONOUS READ 0.80 4 SYNCHRONOUS READ 72.60 363 SYNCHRONOUS READ 73.40 SEQ. PREFETCH REQS 0.00 0 SEQ. PREFETCH REQS 0.00 0 SEQ. PREFETCH REQS 0.00 LIST PREFETCH REQS 0.00 0 LIST PREFETCH REQS 0.00 0 LIST PREFETCH REQS 0.00 DYN. PREFETCH REQS 0.20 1 DYN. PREFETCH REQS 0.00 0 DYN. PREFETCH REQS 0.20 PAGES READ ASYNCHR. 0.60 3 PAGES READ ASYNCHR. 0.00 0 PAGES READ ASYNCHR. 0.60 HPOOL WRITES 0.00 0 HPOOL WRITES 0.00 0 HPOOL WRITES 0.00 HPOOL WRITES-FAILED 0.00 0 HPOOL WRITES-FAILED 0.00 0 HPOOL WRITES-FAILED 0.00 PAGES READ ASYN-HPOOL 0.00 0 PAGES READ ASYN-HPOOL 0.00 0 PAGES READ ASYN-HPOOL 0.00 HPOOL READS 0.00 0 HPOOL READS 0.00 0 HPOOL READS 0.00 HPOOL READS-FAILED 0.00 0 HPOOL READS-FAILED 0.00 0 HPOOL READS-FAILED 0.00

SAMPLE3 VALUE SAMPLE3 TIMES SAMPLE3 AVERAGE TIME AVG.EV TIME/E ------------------ ------------------ ------------------ ------------ ------------------ ------------ ------ -------- TYPE PACKAGE ELAP-CL7 TIME-AVG 3:29.976568 LOCK/LATCH 1:28.359418 55.80 1.58 CPU TIME 1.660909 SYNCHRONOUS I/O 3.145753 71.20 0.04 LOCATION DB2G AGENT 1.660909 OTHER READ I/O 0.000000 0.00 COLLECTION ID COLL000 PAR.TASKS 0.000000 OTHER WRITE I/O 0.000000 0.00 PROGRAM NAME SAMPLE3 SUSPENSION-CL8 2:29.432129 SERV.TASK SWITCH 57.926959 15.00 3.86 AGENT 2:29.432129 ARCH.LOG(QUIESCE) 0.000000 0.00 OCCURRENCES 5 PAR.TASKS 0.000000 ARCHIVE LOG READ 0.000000 0.00 SQL STMT - AVERAGE 49.60 NOT ACCOUNTED 58.883529 STORED PROC SCHED 0.000000 0.00 SQL STMT - TOTAL 248 AVG.DB2 ENTRY/EXIT 99.20 UDF SCHEDULE 0.000000 0.00 STOR PROC EXECUTED 0 DB2 ENTRY/EXIT 496 DRAIN LOCK 0.000000 0.00 UDF EXECUTED 0 CLAIM RELEASE 0.000000 0.00 USED BY STOR PROC 0 CPU SERVICE UNITS 19104.60 PAGE LATCH 0.000000 0.00 USED BY UDF 0 AGENT 19104.60 NOTIFY MESSAGES 0.000000 0.00 USED BY TRIGGER 0 PAR.TASKS 0.00 GLOBAL CONTENTION 0.000000 0.00 SUCC AUTH CHECK 0 TOTAL CL8 SUSPENS. 2:29.432129 142.00 1.05


*** GRAND TOTAL *** ELAPSED TIME DISTRIBUTION CLASS 2 TIME DISTRIBUTION ---------------------------------------------------------------- ------------------------------------------------------------ APPL !============================================> 89% CPU !> 1% DB2 !=> 3% NOTACC !==============> 28% SUSP !====> 8% SUSP !===================================> 71%

AVERAGE APPL(CL.1) DB2 (CL.2) IFI (CL.5) CLASS 3 SUSPENSIONS AVERAGE TIME AV.EVENT HIGHLIGHTS ------------ ---------- ---------- ---------- -------------------- ------------ -------- -------------------------- ELAPSED TIME 14:54.4918 1:35.51485 N/P LOCK/LATCH(DB2+IRLM) 40.163372 25.36 #OCCURRENCES : 11 NONNESTED 14:54.4918 1:35.51485 N/A SYNCHRON. I/O 1.429888 32.36 #ALLIEDS : 11 STORED PROC 0.000000 0.000000 N/A DATABASE I/O 1.429888 32.36 #ALLIEDS DISTRIB: 0 UDF 0.000000 0.000000 N/A LOG WRITE I/O 0.000000 0.00 #DBATS : 0 TRIGGER 0.000000 0.000000 N/A OTHER READ I/O 0.000000 0.00 #DBATS DISTRIB. : 0 OTHER WRTE I/O 0.000000 0.00 #NO PROGRAM DATA: 4 CPU TIME 2.002305 0.756192 N/P SER.TASK SWTCH 26.434087 12.73 #NORMAL TERMINAT: 6 AGENT 2.002305 0.756192 N/A UPDATE COMMIT 18.286116 1.00 #ABNORMAL TERMIN: 5 NONNESTED 2.002305 0.756192 N/P OPEN/CLOSE 0.035584 0.09 #CP/X PARALLEL. : 0 STORED PRC 0.000000 0.000000 N/A SYSLGRNG REC 0.010161 0.18 #IO PARALLELISM : 0 UDF 0.000000 0.000000 N/A EXT/DEL/DEF 8.064274 6.00 #INCREMENT. BIND: 0 TRIGGER 0.000000 0.000000 N/A OTHER SERVICE 0.037953 5.45 #COMMITS : 7


PAR.TASKS 0.000000 0.000000 N/A ARC.LOG(QUIES) 0.000000 0.00 #ROLLBACKS : 5 ARC.LOG READ 0.000000 0.00 #SVPT REQUESTS : 0 SUSPEND TIME N/A 1:08.02735 N/A STOR.PRC SCHED 0.000000 0.00 #SVPT RELEASE : 0 AGENT N/A 1:08.02735 N/A UDF SCHEDULE 0.000000 0.00 #SVPT ROLLBACK : 0 PAR.TASKS N/A 0.000000 N/A DRAIN LOCK 0.000000 0.00 MAX SQL CASC LVL: 0 CLAIM RELEASE 0.000000 0.00 UPDATE/COMMIT : 0.08 NOT ACCOUNT. N/A 26.731315 N/A PAGE LATCH 0.000000 0.00 SYNCH I/O AVG. : 0.044182 DB2 ENT/EXIT N/A 49.27 N/A NOTIFY MSGS 0.000000 0.00 EN/EX-STPROC N/A 0.00 N/A GLOBAL CONTENTION 0.000000 0.00 EN/EX-UDF N/A 0.00 N/A COMMIT PH1 WRITE I/O 0.000000 0.00 DCAPT.DESCR. N/A N/A N/P ASYNCH IXL REQUESTS 0.000000 0.00 LOG EXTRACT. N/A N/A N/P TOTAL CLASS 3 1:08.027347 70.45

GLOBAL CONTENTION L-LOCKS AVERAGE TIME AV.EVENT GLOBAL CONTENTION P-LOCKS AVERAGE TIME AV.EVENT ------------------------------------- ------------ -------- ------------------------------------- ------------ -------- L-LOCKS 0.000000 0.00 P-LOCKS 0.000000 0.00 PARENT (DB,TS,TAB,PART) 0.000000 0.00 PAGESET/PARTITION 0.000000 0.00 CHILD (PAGE,ROW) 0.000000 0.00 PAGE 0.000000 0.00 OTHER 0.000000 0.00 OTHER 0.000000 0.00


*** GRAND TOTAL *** SQL DML AVERAGE TOTAL SQL DCL TOTAL SQL DDL CREATE DROP ALTER LOCKING AVERAGE -------------- -------- ---------- ------ ------ ------ ---------------------- -------- ---------- SELECT 0.00 0 LOCK TABLE 0 TABLE 0 0 0 TIMEOUTS 0.00 INSERT 0.09 1 GRANT 0 CRT TTABLE 0 N/A N/A DEADLOCKS 0.00 UPDATE 0.00 0 REVOKE 0 DCL TTABLE 0 N/A N/A ESCAL.(SHARED) 0.00 DELETE 0.00 0 SET CURR.SQLID 0 AUX TABLE 0 N/A N/A ESCAL.(EXCLUS) 0.00 SET HOST VAR. 248 INDEX 0 1 0 MAX PG/ROW LOCKS HELD 0.64 DESCRIBE 0.00 0 SET CUR.DEGREE 0 TABLESPACE 0 0 0 LOCK REQUEST 44.36 DESC.TBL 0.00 0 SET RULES 0 DATABASE 0 0 0 UNLOCK REQUEST 41.27 PREPARE 0.09 1 SET CURR.PATH 0 STOGROUP 0 0 0 QUERY REQUEST 0.00 OPEN 0.00 0 SET CURR.PREC. 0 SYNONYM 0 0 N/A CHANGE REQUEST 0.82 FETCH 0.00 0 CONNECT TYPE 1 0 VIEW 0 0 N/A OTHER REQUEST 0.00 CLOSE 0.00 0 CONNECT TYPE 2 0 ALIAS 0 0 N/A LOCK SUSPENSIONS 24.91 SET CONNECTION 0 PACKAGE N/A 0 N/A IRLM LATCH SUSPENSIONS 0.00 RELEASE 0 PROCEDURE 0 0 0 OTHER SUSPENSIONS 0.00 DML-ALL 0.18 2 CALL 0 FUNCTION 0 0 0 TOTAL SUSPENSIONS 24.91 ASSOC LOCATORS 0 TRIGGER 0 0 N/A ALLOC CURSOR 0 DIST TYPE 0 0 N/A HOLD LOCATOR 0 FREE LOCATOR 0 TOTAL 0 1 0 DCL-ALL 248 RENAME TBL 0 COMMENT ON 0 LABEL ON 0

NORMAL TERM. AVERAGE TOTAL ABNORMAL TERM. TOTAL IN DOUBT TOTAL DRAIN/CLAIM AVERAGE --------------- -------- -------- ----------------- -------- -------------- -------- -------------- -------- ---- NEW USER 0.00 0 APPL.PROGR. ABEND 5 APPL.PGM ABEND 0 DRAIN REQUESTS 0.09 DEALLOCATION 0.55 6 END OF MEMORY 0 END OF MEMORY 0 DRAIN FAILED 0.00 APPL.PROGR. END 0.00 0 RESOL.IN DOUBT 0 END OF TASK 0 CLAIM REQUESTS 3.18 RESIGNON 0.00 0 CANCEL FORCE 0 CANCEL FORCE 0 CLAIM FAILED 0.00 DBAT INACTIVE 0.00 0 RRS COMMIT 0.00 0

DATA CAPTURE AVERAGE TOTAL DATA SHARING AVERAGE TOTAL QUERY PARALLELISM AVERAGE ----------------- -------- -------- ------------------- -------- -------- ---------------------------- -------- ---- IFI CALLS MADE N/P N/P GLOBAL CONT RATE(%) N/C N/A MAXIMUM MEMBERS USED N/A RECORDS CAPTURED N/P N/P FALSE CONT RATE(%) N/C N/A MAXIMUM DEGREE N/A LOG RECORDS READ N/P N/P L-LOCKS XES RATE(%) N/C N/A GROUPS EXECUTED 0.00 ROWS RETURNED N/P N/P LOCK REQ - PLOCKS 0.00 0 RAN AS PLANNED 0.00 RECORDS RETURNED N/P N/P UNLOCK REQ - PLOCKS 0.00 0 RAN REDUCED 0.00 DATA DESC. RETURN N/P N/P CHANGE REQ - PLOCKS 0.00 0 ONE DB2-COORDINATOR = NO 0.00 TABLES RETURNED N/P N/P LOCK REQ - XES 0.00 0 ONE DB2-ISOLATION LEVEL 0.00 DESCRIBES N/P N/P UNLOCK REQ - XES 0.00 0 ONE DB2-DCL TEMPORARY TABLE 0.00 CHANGE REQ - XES 0.00 0 SEQUENTIAL-CURSOR 0.00 SUSPENDS - IRLM 0.00 0 SEQUENTIAL-NO ESA SORT 0.00 SUSPENDS - XES 0.00 0 SEQUENTIAL-NO BUFFER 0.00 SUSPENDS - FALSE 0.00 0 SEQUENTIAL-ENCLAVE SERVICES 0.00 INCOMPATIBLE LOCKS 0.00 0 MEMBER SKIPPED (%) N/C NOTIFY MSGS SENT 0.00 0 DISABLED BY RLF 0.00 REFORM PARAL-CONFIG 0.00 REFORM PARAL-NO BUF 0.00

1 LOCATION: DB2G DB2 PERFORMANCE MONITOR (V7) PAGE: 2-20 GROUP: N/P ACCOUNTING REPORT - LONG REQUESTED FROM: NOT SPECIFIED MEMBER: N/P TO: NOT SPECIFIED SUBSYSTEM: DB2G ORDER: PLANNAME INTERVAL FROM: 03/18/02 22:36:2


DB2 VERSION: V7 SCOPE: MEMBER TO: 03/18/02 23:19:3

*** GRAND TOTAL *** STORED PROCEDURES AVERAGE TOTAL UDF AVERAGE TOTAL TRIGGERS AVERAGE TOTAL ----------------- -------- -------- --------- -------- -------- ----------------- -------- -------- CALL STATEMENTS 0.00 0 EXECUTED 0.00 0 STATEMENT TRIGGER 0.00 0 ABENDED 0.00 0 ABENDED 0.00 0 ROW TRIGGER 0.00 0 TIMED OUT 0.00 0 TIMED OUT 0.00 0 SQL ERROR OCCUR 0.00 0 REJECTED 0.00 0 REJECTED 0.00 0

LOGGING AVERAGE TOTAL ROWID AVERAGE TOTAL RID LIST AVERAGE TOTAL ------------------- -------- -------- ------------- -------- -------- ------------------- -------- -------- LOG RECORDS WRITTEN 13.73 151 DIRECT ACCESS 0.00 0 USED 0.00 0 TOT BYTES WRITTEN 6169.82 67868 INDEX USED 0.00 0 FAIL-NO STORAGE 0.00 0 TS SCAN USED 0.00 0 FAIL-LIMIT EXCEEDED 0.00 0

AVERAGE SU CLASS 1 CLASS 2 DYNAMIC SQL STMT AVERAGE TOTAL MISCELLANEOUS AVERAGE TOTAL ------------ -------------- -------------- ------------------ -------- -------- ------------------- -------- ------- CPU 23031.36 8697.73 REOPTIMIZATION 0.00 0 MAX STOR LOB VALUES 46547.36 512021 AGENT 23031.36 8697.73 NOT FOUND IN CACHE 0.00 0 NONNESTED 23031.36 8697.73 FOUND IN CACHE 0.00 0 STORED PRC 0.00 0.00 IMPLICIT PREPARES 0.00 0 UDF 0.00 0.00 PREPARES AVOIDED 0.00 0 TRIGGER 0.00 0.00 STMT INVALID (MAX) 0.00 0 PAR.TASKS 0.00 0.00 STMT INVALID (DDL) 0.00 0

BP0 AVERAGE TOTAL BP11 AVERAGE TOTAL BP12 AVERAGE --------------------- -------- -------- --------------------- -------- -------- --------------------- -------- ----- BPOOL HIT RATIO (%) 98.96 N/A BPOOL HIT RATIO (%) 42.86 N/A BPOOL HIT RATIO (%) 100.00 HPOOL HIT RATIO (%) 0.00 N/A HPOOL HIT RATIO (%) 0.00 N/A HPOOL HIT RATIO (%) N/C GETPAGES 48.17 289 GETPAGES 1.40 7 GETPAGES 0.80 GETPAGES-FAILED 0.00 0 GETPAGES-FAILED 0.00 0 GETPAGES-FAILED 0.00 BUFFER UPDATES 4.83 29 BUFFER UPDATES 0.60 3 BUFFER UPDATES 0.80 SYNCHRONOUS WRITE 0.00 0 SYNCHRONOUS WRITE 0.20 1 SYNCHRONOUS WRITE 0.00 SYNCHRONOUS READ 0.00 0 SYNCHRONOUS READ 0.80 4 SYNCHRONOUS READ 0.00 SEQ. PREFETCH REQS 0.00 0 SEQ. PREFETCH REQS 0.00 0 SEQ. PREFETCH REQS 0.00 LIST PREFETCH REQS 0.00 0 LIST PREFETCH REQS 0.00 0 LIST PREFETCH REQS 0.00 DYN. PREFETCH REQS 0.17 1 DYN. PREFETCH REQS 0.00 0 DYN. PREFETCH REQS 0.00 PAGES READ ASYNCHR. 0.50 3 PAGES READ ASYNCHR. 0.00 0 PAGES READ ASYNCHR. 0.00 HPOOL WRITES 0.00 0 HPOOL WRITES 0.00 0 HPOOL WRITES 0.00 HPOOL WRITES-FAILED 0.00 0 HPOOL WRITES-FAILED 0.00 0 HPOOL WRITES-FAILED 0.00 PAGES READ ASYN-HPOOL 0.00 0 PAGES READ ASYN-HPOOL 0.00 0 PAGES READ ASYN-HPOOL 0.00 HPOOL READS 0.00 0 HPOOL READS 0.00 0 HPOOL READS 0.00 HPOOL READS-FAILED 0.00 0 HPOOL READS-FAILED 0.00 0 HPOOL READS-FAILED 0.00


*** GRAND TOTAL *** TOT4K AVERAGE TOTAL BP32K AVERAGE TOTAL TOTAL AVERAGE TO --------------------- -------- -------- --------------------- -------- -------- --------------------- -------- ----- BPOOL HIT RATIO (%) 97.67 N/A BPOOL HIT RATIO (%) 97.79 N/A BPOOL HIT RATIO (%) 97.78 HPOOL HIT RATIO (%) 0.00 N/A HPOOL HIT RATIO (%) 0.00 N/A HPOOL HIT RATIO (%) 0.00 GETPAGES 50.00 300 GETPAGES 3278.00 16390 GETPAGES 2781.67 16 GETPAGES-FAILED 0.00 0 GETPAGES-FAILED 0.00 0 GETPAGES-FAILED 0.00 BUFFER UPDATES 6.00 36 BUFFER UPDATES 3210.20 16051 BUFFER UPDATES 2681.17 16 SYNCHRONOUS WRITE 0.17 1 SYNCHRONOUS WRITE 0.00 0 SYNCHRONOUS WRITE 0.17 SYNCHRONOUS READ 0.67 4 SYNCHRONOUS READ 72.60 363 SYNCHRONOUS READ 61.17 SEQ. PREFETCH REQS 0.00 0 SEQ. PREFETCH REQS 0.00 0 SEQ. PREFETCH REQS 0.00 LIST PREFETCH REQS 0.00 0 LIST PREFETCH REQS 0.00 0 LIST PREFETCH REQS 0.00 DYN. PREFETCH REQS 0.17 1 DYN. PREFETCH REQS 0.00 0 DYN. PREFETCH REQS 0.17 PAGES READ ASYNCHR. 0.50 3 PAGES READ ASYNCHR. 0.00 0 PAGES READ ASYNCHR. 0.50 HPOOL WRITES 0.00 0 HPOOL WRITES 0.00 0 HPOOL WRITES 0.00 HPOOL WRITES-FAILED 0.00 0 HPOOL WRITES-FAILED 0.00 0 HPOOL WRITES-FAILED 0.00 PAGES READ ASYN-HPOOL 0.00 0 PAGES READ ASYN-HPOOL 0.00 0 PAGES READ ASYN-HPOOL 0.00 HPOOL READS 0.00 0 HPOOL READS 0.00 0 HPOOL READS 0.00 HPOOL READS-FAILED 0.00 0 HPOOL READS-FAILED 0.00 0 HPOOL READS-FAILED 0.00

ALL PROG VALUE ALL PROG TIMES ALL PROG AVERAGE TIME AVG.EV TIME/E ------------------ ------------------ ------------------ ------------ ------------------ ------------ ------ -------- TYPE PACKAGE ELAP-CL7 TIME-AVG 2:30.091872 LOCK/LATCH 1:03.113870 39.86 1.58 CPU TIME 1.187084 SYNCHRONOUS I/O 2.246966 50.86 0.04 LOCATION 'BLANK' AGENT 1.187084 OTHER READ I/O 0.000000 0.00 COLLECTION ID 'BLANK' PAR.TASKS 0.000000 OTHER WRITE I/O 0.000000 0.00 PROGRAM NAME ALL PROG SUSPENSION-CL8 1:46.842769 SERV.TASK SWITCH 41.481932 11.14 3.72 AGENT 1:46.842769 ARCH.LOG(QUIESCE) 0.000000 0.00 OCCURRENCES 7 PAR.TASKS 0.000000 ARCHIVE LOG READ 0.000000 0.00 SQL STMT - AVERAGE 36.14 NOT ACCOUNTED 42.062020 STORED PROC SCHED 0.000000 0.00 SQL STMT - TOTAL 253 AVG.DB2 ENTRY/EXIT 73.14 UDF SCHEDULE 0.000000 0.00 STOR PROC EXECUTED 0 DB2 ENTRY/EXIT 512 DRAIN LOCK 0.000000 0.00


UDF EXECUTED 0 CLAIM RELEASE 0.000000 0.00 USED BY STOR PROC 0 CPU SERVICE UNITS 13654.29 PAGE LATCH 0.000000 0.00 USED BY UDF 0 AGENT 13654.29 NOTIFY MESSAGES 0.000000 0.00 USED BY TRIGGER 0 PAR.TASKS 0.00 GLOBAL CONTENTION 0.000000 0.00 SUCC AUTH CHECK 0 TOTAL CL8 SUSPENS. 1:46.842769 101.86 1.04

ACCOUNTING REPORT COMPLETE

A.3 Using file reference variablesYou can use the file reference variable as input to the extender UDFs. In this section we have included an example of loading a video clip from a file on the workstation which is stored as a BLOB column at the host.

The table and columns are enabled to handle AVI extenders. This is shown in Example A-1.

Example: A-1 Enabled Extender tables

get extender status in mmdbsys.employee

EXTENDER TABLESPACE COLUMN -------- ----------------------------------- ------------------- DB2VIDEO MMDBSYS.EMPLOYAD,,MMDBSYS.EMPLOYVL, VIDEO DB2IMAGE MMDBSYS.DB2EXTND,,MMDBSYS.DB2LXTND, PICTURE DB2AUDIO MMDBSYS.EMPLOYAA,,MMDBSYS.EMPLOYAL, AUDIO DMB0025I: Table "mmdbsys.employee" is enabled for "3" extenders. db2ext =>

We tested the insert using DB2 Connect command center. The results from the connect statement are:

Database Connection InformationDatabase server = DB2 OS/390 7.1.1SQL authorization ID = PAOLOR3Local database alias = DB2G

We issued an INSERT from a client file to a BLOB column at the host using a relative file name. Example A-2 shows the command and the results.

Example: A-2 Use of file reference

--------------- Command Entered ---------------------------------------------insert into mmdbsys.employee (name, video) values ('XT video',DB2VIDEO(CURRENT SERVER, 'nitecry.avi', 'AVI', 1,'test'))

-----------------------------------------------------------------------------DB20000I The SQL command completed successfully

If you specify a relative file name, the extenders will use the directory specifications in various client and server environment variables as a search path to resolve the file name. In our example the path to the files stored on the Window NT workstation was setup with the installation of the AVI extender software.


Environment variable description � DB2MMPATH

It is used to resolve source file name for store, retrieve, and update operations. Our path is: c:\dmb\samples

� DB2MMSTORE

It is used to resolve target file name for store and update operations. Our path is: c:\dmb\samples

� DB2MMTEMP

It is used to resolve file name for operations that create temporary files. Our path is: c:\dmb\tmp

These parameters are set up in My Computer->Settings->Control Panel->Environment.


Appendix B. Unicode implementation

DB2 V7 supports UNICODE conversion from the host encoding scheme to UNICODE 367. DB2 utilizes OS/390 services for the code conversion. This appendix describes the procedures to install and activate the 1140 (Euro English) to UNICODE 367 conversion.

A full description of the OS/390 UNICODE support can be found at:

http://www.ibm.com/servers/s390/os390/bkserv/latest/v2r10unicode.html

Here we show as an example only the CCSID 1140 to 367 conversion installation.

B


http://www.ibm.com/servers/s390/os390/bkserv/latest/v2r10unicode.html

B.1 Generate the conversion tableThe JCL in Example B-1 creates the conversion table and output to DD SYSIMG. Please note that the ER option must be supplied to the CONVERSION command. The ER option contains two characters, representing the technique-search-order; respectively, E for enforced subset and R for roundtrip.

Example: B-1 JCL to create the conversion table

//CUNJIUTL JOB (POK,999),'UNICODE',MSGLEVEL=(1,1),MSGCLASS=T, // CLASS=A,NOTIFY=&SYSUID //******************************************************************* //* * //* LICENSED MATERIALS - PROPERTY OF IBM * //* * //* 5647-A01 * //* * //* (C) COPYRIGHT IBM CORP. 2000 * //* * //* STATUS = HUNI2A0 * //* * //******************************************************************* //* * //* IMAGE GENERATOR * //* CHANGED TO USE ER !!! * //******************************************************************* //CUNMIUTL EXEC PGM=CUNMIUTL //STEPLIB DD DSN=SYS1.LINKLIB,DISP=SHR //SYSPRINT DD SYSOUT=* //TABIN DD DISP=SHR,DSN=SYS1.SCUNTBL //SYSIMG DD DSN=PAOLOR1.IMAGES(CUNIMG01),DISP=SHR //SYSIN DD * /******************************************** * INPUT STATEMENTS FOR THE IMAGE GENERATOR * ********************************************/ CASE NORMAL; /* ENABLE TOUPPER AND TOLOWER */ CONVERSION 1140,367,ER; /* EBCDIC EURO -> UNICODE */ CONVERSION 367,1140,ER; /* UNICODE -> EBCDIC EURO */ /*

The output of the job is shown in Example B-2. Please note that the job creates the conversion table indirectly, since no direct conversion from 1140 to 367 is currently available.

Example: B-2 Output from the generation job

CUN1000I OS/390 SUPPORT FOR UNICODE VERSION 2.8.0 CUN1001I PROCESSING STARTED ON 06/25/2001 AT 14:21:56 Source Listing ----+----1----+----2----+----3----+----4----+----5----+----6---- 1 2 /******************************************** 3 * INPUT STATEMENTS FOR THE IMAGE GENERATOR * 4 ********************************************/ 5 6 CASE NORMAL; /* ENABLE TOUPPER AND TOLOWER */


7 8 CONVERSION 1140,367,ER; /* EBCDIC EURO -> UNICODE */ 9 CONVERSION 367,1140,ER; /* UNICODE -> EBCDIC EURO */ 10 Statement Report --+----1----+----2----+----3----+----4----+----5----+----6---- 1 CONVERSION 1140,367,ER; CUN0999I NO TABLE FOUND. GENERATING A FORCED INDIRECT /* 01140-00367-ER */ /* 01140-01200-R using CUNRN5PH */ /* 01200-00367-X using CUNXPGB0 */ 2 CONVERSION 367,1140,ER; CUN0999I NO TABLE FOUND. GENERATING A FORCED INDIRECT /* 00367-01140-ER */ /* 00367-01200-R using CUNRB0PG */ /* 01200-01140-E using CUNEPHN5 */ 3 CASE NORMAL; /* to-upper using CUNANUUP */ /* to-lower using CUNANULO */ CUN1014I INPUT READ 10 RECORDS CUN1015I STATEMENTS PROCESSED 3 CUN1016I STATEMENTS FLAGGED 0 CUN1017I GENERATED IMAGE SIZE 98 PAGES CUN1002I PROCESSING ENDED. HIGHEST RETURN CODE WAS 0

B.2 Activate the UNICODE tableThese are the steps to activate the UNICODE conversion table:

� Copy the conversion table (the highlighted module in Example B-1 on page 148) into SYS1.PARMLIB as CUNIMG01.

� Create a new member in SYS1.PARMLIB, member name CUNUNI01. The contents of CUNUNI01 are shown in Example B-3.

Example: B-3 Contents of CUNUNI01 in SYS1.PARMLIB

/**********************************************************/ /* */ /* CUNUNIXX - UNICODE CONVERSION CONTROL PARAMETERS */ /* */ /**********************************************************/ /* ESTABLISH A NEW ENVIRONMENT */ /**********************************************************/ /* REQUIRED KEYWORD REALSTORAGE */ /* MAXIMAL USED PAGES OF REAL STORAGE, MIN=0 MAX=524287 */ /* WHERE 0 MEANS NO EXPLICITE LIMIT (=524287) */ /**********************************************************/ REALSTORAGE 51200; /* E.G. 200 MB */ /**********************************************************/ /* REQUIRED KEYWORD IMAGE WITH */ /* REQUIRED PARAMETER: MEMBER NAME */ /* THIS MEMBER MUST BE PLACED IN A DATA SET FROM THE */ /* LOGICAL PARMLIB CONCATENATION (DEF'D IN LOADXX) */ /**********************************************************/ IMAGE CUNIMG01;

Appendix B. Unicode implementation 149

� Activate the new UNICODE table with command T UNI=01. The SYSLOG output is reported in Example B-4.

Example: B-4 Activate the new UNICODE table

T UNI=01 IEE252I MEMBER CUNUNI01 FOUND IN SYS1.PARMLIB 537 CUN2036I INACTIVE CONVERSION ENVIRONMENT (UNICODE1) WILL BE DELETED. ARE YOU SURE? (Y/N) R 537,Y IEE600I REPLY TO 537 IS;Y CUN2038I INACTIVE CONVERSION ENVIRONMENT (UNICODE1) 282 WAS SUCCESSFULLY DELETED CUN2020I START LOADING CONVERSION IMAGE CUNIMG01 IEE252I MEMBER CUNIMG01 FOUND IN SYS1.PARMLIB CUN2022I LOADING CONVERSION IMAGE CUNIMG01 FINISHED: 401434 BYTES LOADED CUN2034I SET UNI COMMAND SUCCESSFULLY EXECUTED IEE536I UNI VALUE 01 NOW IN EFFECT

� Verify the UNICODE table activation with command D UNI,ALL. The SYSLOG output is reported in Example B-5.

Example: B-5 UNI=ALL output in SYSLOG

D UNI,ALL IEF196I IEF237I 3D30 ALLOCATED TO SYS00108 IEF196I IEF285I SYS1.LINKLIB KEPT IEF196I IEF285I VOL SER NOS= Z01RE1. CUN3000I 14.26.27 UNI DISPLAY 303 ENVIRONMENT: CREATED 06/22/2001 AT 10.02.01 MODIFIED 06/25/2001 AT 14.24.25 IMAGE CREATED 06/25/2001 AT 14.21.56 SERVICE: CUNMCNV CUNMCASE STORAGE: ACTIVE 98 PAGES INACTIVE 98 PAGES SINCE 06/25/2001 AT 14.24.25 LIMIT 51200 PAGES CASECONV: NORMAL CONVERSION: 01140-00367-ER 00367-01140-ER


Appendix C. Recent maintenance

In this appendix we provide feedback on the implementation of the environment and products. It consists of a list of the opened requirements and the APARs recommended.

� Open requirements� APARs

C


C.1 RequirementsIn Table C-1 we briefly describe the current restrictions encountered during our tests. Their solutions have been submitted as requirements with the Fully Integrated Tool Set Marketing/Office requirements (FITSMO) application, a set of Lotus Notes databases, and a Web Interface, used to gather requirements from external customers and business partners to improve products and policies.

Table C-1 Requirements

C.2 LOB related maintenanceIn Table C-2 we briefly describe the DB2 LOB related APARS that we came across during our project with DB2 V7.

Table C-2 DB2 V7 LOB related APARs

Number Area Description

MR0322024656 Ease of use A program needs to be created to run on the ‘connected workstation’ to push the data up to the mainframe and insert it if you want to use the file reference variables.

MR0322023113 Explain and auxiliary tables

If an SQL statement visually accesses the base table (when selecting a LOB column), the explain output does not show any access to the auxiliary table nor the auxiliary index. Only the access path for the base table is shown. This may confuse those people who do not know that a base table and a LOB column is accessed by the SQL statement.

MR032605920 Acceptability AUXW state after recovery - only reset by REPAIR utility and program. Needs to optionally reset at CHECK LOB time, with LOB values set to null. Also EXCEPTION tables like for RI would help.

MR0322021925 REORG space reclaim

REORG to do normal REORG, i.e. release space (remove deleted objects physically).

MR0322023637 Accessibility LOB Load to support > 32 KB (maybe in a different format, n times 32,760 bytes would build one row).

MR0322023750 Usability Partitioning of LOB table spaces would correspond to base table design.

MR0322024927 Usability POSSTR enhancement to support not just the 1st but the nth occurrence of a string.

MR0322026439 Performance Compression on LOBs (maybe only CLOBs and DBCLOBs).

APAR Area Description

PQ42864 Runstats PTF UQ54223 available.

PQ54511 DB2 Extenders UQ63855 being prepared. It Enables AVI Extenders with DB2 V7.

PQ46137 CHECK DATA with LOBs

It fixes several important problems and provides better messages. PTF UQ64673 is available for DB2 V7; UQ64672 for DB2 V6.


Note:(1) The APAR is due to a remote possibility of loss of integrity at write time for stripes involvingpages larger than 4 KB. If you want to use striping with LOBs this restriction is not penalizingperformance and is not in contrast with the general recommendation of using 32 KB pages forlarge LOBs. Our opinion, supported by the lab measurements, is that the very small penalty ina little extra CPU time when concatenating 4 KB pages is really negligible, while I/O time re-mains the same. So it might be worthwhile changing to 4 KB pages if you want to exploit striping,even for large LOBs. Furthermore, a large majority of LOB users seem to have either smallLOBs or a large variation in size, therefore justifying 4 KB pages in any case.

PQ53571(1) DB2 striped data set enhancements

UQ64357 for DB2 V7. VSAM striping is allowed on table space and index space with 4 KB page size, and disallowed on those with 8, 16, or 32 KB page sizes. Preformat quantity is changed with striping.

PQ58306 LOB utilities UQ64118. It fixes errors due to inconsistent LOB data after forward log recovery of a LOB table space defined LOG NO.

PQ59820 LOB support LOAD performance and bad page number. PTF UQ65857is available for DB2 V7, UQ65856 is available for DB2 V6.

APAR Area Description

Appendix C. Recent maintenance 153

Appendix D. Additional material

This redbook refers to additional material that can be downloaded from the Internet as described below.

Locating the Web materialThe Web material associated with this redbook is available in softcopy on the Internet from the IBM Redbooks Web server. Point your Web browser to:

ftp://www.redbooks.ibm.com/redbooks/SG246571

Alternatively, you can go to the IBM Redbooks Web site at:

ibm.com/redbooks

Select the Additional materials and open the directory that corresponds with the redbook form number, SG24-6571.

Using the Web materialThe additional Web material that accompanies this redbook includes the following files:

File name DescriptionLOBDDL.TXT DDL for creating the LOB environment needed by the samplesBIND.TXT BIND PACKAGE and BIND PLAN for sample programsSAMPLE1.TXT Sample program for inserting LOB data using one large host variableSAMPLE1.JCL JCL to run program SAMPLE1SAMPLE2.TXT Sample program for inserting LOB data using one large host variable

and a single locator chainSAMPLE2.JCL JCL to run program SAMPLE2SAMPLE3.TXT Sample program for inserting LOB data using one large host variable

and two locator chainsSAMPLE3.JCL JCL to run program SAMPLE3SAMPLE4.TXT Sample program for unloading LOB data using one large host variableSAMPLE4.JCL JCL to run program SAMPLE4

D


ftp://www.redbooks.ibm.com/redbooks/




SAMPLE5.TXT Sample program for unloading LOB data using locators and a hostvariable of 1 MB

SAMPLE5.JCL JCL to run program SAMPLE5SAMPLE6.TXT Sample program for deleting a special part of a LOB using locatorsSAMPLE6.JCL JCL to run program SAMPLE6SAMPLE7.TXT Sample program for updating a special part of a LOB using locatorsSAMPLE7.JCL JCL to run program SAMPLE7

All the above files are contained in: LOBSAMPLES.zip Zipped Code Samples

System requirements for downloading the Web materialThe following system configuration is recommended:

Hard disk space: 8 MB minimumOperating System: Windows 95 or NT or 2000Processor: Intel 386 or higherMemory: 16 MB

How to use the Web materialCreate a subdirectory (folder) on your workstation, and unzip the contents of the Web material zip file into this folder.


ronyms

AIX Advanced Interactive eXecutive from IBM

APAR authorized program analysis report

ARM automatic restart manager

ASCII American National Standard Code for Information Interchange

BLOB binary large object

CCA client configuration assistant

CCSID coded character set identifier

CD compact disk

CEC central electronics complex

CF coupling facility

CFCC coupling facility control code

CFRM coupling facility resource management

CLI call level interface

CLOB character large object

CLP command line processor

CPU central processing unit

CSA common storage area

CTT created temporary table

DAD document access definition

DASD direct access storage device

DB2 PM DB2 performance monitor

DBAT database access thread

DBCLOB double byte character large object

DBD database descriptor

DBID database identifier

DBMS database management system

DBRM database request module

DCL data control language

DDCS distributed database connection services

DDF distributed data facility

DDL data definition language

DLL dynamic load library manipulation language

DML data manipulation language

DNS domain name server

DRDA distributed relational database architecture

Abbreviations and ac

© Copyright IBM Corp. 2002

DSC dynamic statement cache, local or global

DTT declared temporary tables

EA extended addressability

EBCDIC extended binary coded decimal interchange code

ECS enhanced catalog sharing

ECSA extended common storage area

EDM environment descriptor management

ERP enterprise resource planning

ESA Enterprise Systems Architecture

ESS Enterprise Storage Server

ETR external throughput rate, an elapsed time measure, focuses on system capacity

FDT functional track directory

FTP File Transfer Program

GB gigabyte (1,073,741,824 bytes)

GBP group buffer pool

GRS global resource serialization

GUI graphical user interface

HA Host adapter

HFS Hierarchical File System

HPJ high performance Java

I/O input/output

IBM International Business Machines Corporation

ICF integrated catalog facility

ICF integrated coupling facility

ICMF internal coupling migration facility

IFCID instrumentation facility component identifier

IFI instrumentation facility interface

IPLA IBM Program Licence Agreement

IRLM internal resource lock manager

IRWW IBM Relational Warehouse Workload

ISPF interactive system productivity facility

ISV independent software vendor

157

ITR internal throughput rate, a processor time measure, focuses on processor capacity

ITSO International Technical Support Organization

IVP installation verification process

JDBC Java Database Connectivity

JFS journaled file systems

JIT Just in time (Java compiler)

JNI Java Native Interface

JVM Java Virtual Machine

KB kilobyte (1,024 bytes)

LCU logical control unit

LOB large object

LPAR Logical Partition

LPL logical page list

LRECL logical record length

LRSN log record sequence number

LVM logical volume manager

MB megabyte (1,048,576 bytes)

MSM Multidimensional Storage Manager

NPI non partitioning index

NVS Non Volatile Storage

ODB object descriptor in DBD

ODBC Open Data Base Connectivity

OLAP Online Analytical Processing

OS/390 Operating System/390

PAV Parallel Access Volume

PDS partitioned data set

PIB parallel index build

PSID pageset identifier

PSP preventive service planning

PTF program temporary fix

PUNC possibly uncommitted

QBIC query by image content

QMF Query Management Facility

RACF Resource Access Control Facility

RBA relative byte address

RECFM record format

RID record identifier

ROT rule of thumb

RR repeatable read

RRS resource recovery services

RRSAF resource recovery services attach facility

RS read stability

RSM Relational Resource Manager

RTS real time statistics

RVA RAMAC Virtual Array

SDK software developers kit

SMIT System Management Interface Tool

SVL IBM Silicon Valley Laboratory

TCB Task control block

USS UNIX System Services

WAS WebSphere Application Service

WLM Workload Manager


Abbreviations and acronyms 159

Related publications

The publications listed in this section are considered particularly suitable for a more detailed discussion of the topics covered in this redbook.

IBM RedbooksFor information on ordering these publications, see “How to get IBM Redbooks” on page 162.

� DB2 for z/OS and OS/390 Version 7 Using the Utilities Suite, SG24-6289

� DB2 for z/OS and OS/390 Version 7 Performance Topics, SG24-6129

� DB2 UDB Server for OS/390 and z/OS Version 7 Presentation Guide, SG24-6121

� DB2 UDB Server for OS/390 Version 6 Technical Update, SG24-6108

� DB2 UDB for OS/390 Version 6 Performance Topics, SG24-5351

� DB2 for z/OS Application Programming Topics, SG24-6300

Other resourcesThese publications are also relevant as further information sources:

� DB2 UDB for OS/390 and z/OS Version 7 Installation Guide, GC26-9936

� DB2 UDB for OS/390 and z/OS Version 7 Command Reference, SC26-9934

� DB2 UDB for OS/390 and z/OS Version 7 Messages and Codes, GC26-9940

� DB2 UDB for OS/390 and z/OS Version 7 Utility Guide and Reference, SC26-9945

� DB2 UDB for OS/390 and z/OS Version 7 Programming Guide and Reference for Java, SC26-9932

� DB2 UDB for OS/390 and z/OS Version 7 Administration Guide, SC26-9931

� DB2 UDB for OS/390 and z/OS Version 7 Application Programming and SQL Guide, SC26-9933

� DB2 UDB for OS/390 and z/OS Version 7 Release Planning Guide, SC26-9943

� DB2 UDB for OS/390 and z/OS Version 7 SQL Reference, SC26-9944

� DB2 UDB for OS/390 and z/OS Version 7 Text Extender Administration and Programming, SC26-9948

� DB2 UDB for OS/390 and z/OS Version 7 Data Sharing: Planning and Administration, SC26-9935

� DB2 UDB for OS/390 and z/OS Version 7 Image, Audio, and Video Extenders, SC26-9947

� DB2 UDB for OS/390 and z/OS Version 7 ODBC Guide and Reference, SC26-9941

� DB2 UDB for OS/390 and z/OS Version 7 XML Extender Administration and Reference, SC26-9949

� DB2 UDB for OS/390 and z/OS Version 7 Diagnosis Guide and Reference, LY37-3740

� DB2 UDB Replication Guide and Reference Version 7, SC26-9920


Referenced Web sitesThese Web sites are also relevant as further information sources:

� DB2 for z/OS and OS/390

http://ibm.com/software/data/db2/os390/

� DB2 for z/OS and OS/390 Version 7 books

http://www.ibm.com/software/data/db2/os390/v7books.html

� UNICODE

http://ibm.com/servers/s390/os390/bkserv/latest/v2r10unicode.html

� DB2 Image, Audio and Video Formats


� Developing Java applications using DB2 Image and Audio Extenders

http://www7b.boulder.ibm.com/dmdd/library/techarticle/cox/0201cox.html

� XML Extender WIZARD


� DAD examples


� Building an XML application, writing a DTD

http://www.ibm.com/developerworks/library/buildappl/writedtd.html

� z/OS UNIX System Services

http://www.s390.ibm.com/products/oe

How to get IBM RedbooksYou can order hardcopy Redbooks, as well as view, download, or search for Redbooks at the following Web site:

ibm.com/redbooks

You can also download additional materials (code samples or diskette/CD-ROM images) from that site.

IBM Redbooks collectionsRedbooks are also available on CD-ROMs. Click the CD-ROMs button on the Redbooks Web site for information about all the CD-ROMs offered, as well as updates and formats.





http://www.s390.ibm.com/products/oe

http://www.ibm.com/developerworks/library/buildappl/writedtd.html



http://www7b.boulder.ibm.com/dmdd/library/techarticle/cox/0201cox.html


http://ibm.com/servers/s390/os390/bkserv/latest/v2r10unicode.html

http://www.ibm.com/software/data/db2/os390/v7books.html

http://ibm.com/software/data/db2/os390/

Index

Numerics00C900Dx 125

AACHKP 113adding a LOB column 30adding a ROWID 69ALTER GROUPBUFFERPOOL 127application programming 3ASCII 15Audio Extender 75aux check pending 113aux table 18, 95AUX WARNING state 24auxiliary column 11, 18auxiliary index 18, 28

DDL 29auxiliary LOB table 11auxiliary table 11auxiliary warning 105AUXW 24, 105AVGSIZE 101

Bbackup and recovery 4base table 11, 17, 20Binary Large Objects 8BIT 97BLOB 8–9browse an image 80buffer pools 23, 25, 125

Ccaching LOB space maps 126CAST 45CCSID 8, 16Character Large Objects 8check constraints 59CHECK DATA 105CHECK-pending 105CHKP 105choosing a page size 25chunk 60client server solution 2CLOB 8COALESCE 45collating 15COLTYPE 92compression 22, 59CONCAT 44concatenating two strings using locators 51COPY 97

© Copyright IBM Corp. 2002

creating a LOB table space 22creating a new table with ROWID 69creating auxiliary table 28creating LOBs 19

DDL statement 20CS 56CURRENT RULES 3, 22, 31

impact on CREATE and DROP 31impact on cursors fetching LOBs 33

CURRENT RULES DB2 33CURRENT RULES STD 33cursor stability 56

DD UNI,ALL 150DAD 86, 88data conversion 14Data Manager 19data propagation 60data sharing 4, 126data space size per system 3data space size per user 3data spaces for LOBs 123database management system xviiDB2 PM 123DB2EXT.CLP 77DB2IMAGE UDT 79DBCLOB 8–9DBMS xviiDBSNDB06 93deferred write threshold 125delete 133deleting a LOB 25deleting part of a LOB 52DFSMS 27displaying data spaces 123displaying LOB objects 30DM 19DMBENVAR DD 76DMBWLM1 76document access definition 86Document Type Definition 88Double Byte Character Large Objects 8DpropR 60DSN8DLPL 94DSNDB06 4DSNTIP7 124DSNZPARM 124DSSIZE 27DTD 88DWQT 125

EENABLE COLUMN 76

163

enabling extenders 77encoding systems 14ER option 148exclusive LOB lock 62Extended Markup Language 9extenders 6

FFOR BIT DATA 9FREE LOCATOR 47FREESPACE 101

GGBPCACHE 27GBPCACHE SYSTEM 27, 126GENERATED ALWAYS 20GENERATED BY DEFAULT 21, 30GRS 136

HHOLD LOCATOR 49host variable 10HURBA 93

IIFCIDs 133Image Copy 24Image Extender 75INCURSOR 94INITSIZE 127insert 132inserting LOBs

using a host-variable 35using locators 36

intent exclusive 65intent-share lock 63internal resource lock manager 67IRLM 67IS 63ISO-10646 15ISOLATION (UR) 25, 64IX 65

JJava stored procedures 4

Llarge objects 6–7LENGTH 92LENGTH2 92LIKE 46list 58list prefetch 58LOAD 94LOAD utility

loading LOBs 34

loading a LOB column 58LOB column 18LOB database 20LOB functions 44LOB indicator 21LOB locators 8LOB lock 62LOB lock serialization 64LOB Manager 19LOB manipulation 52LOB map page 61LOB materialization 120LOB table space 17

DDL statement 22LOBM 19LOBs 8

allocation 26delete 133insert 132performance considerations 127processing 129read performance 130recommendations 136update 132write performance 132

LOBVALA 3, 124LOBVALS 3, 124locators 9, 46

COBOL syntax 11concatenation 39examples 50precompiler conversion 11types 11when to use 49

locators across multiple units of work 49locators and expressions 50locators and materialization 50locking 25, 62, 67locks 59locks with DELETE 65locks with INSERT 65locks with reads 63locks with UPDATE 67LOCKSIZE LOB 25LOCKSIZE TABLESPACE 25logging 3, 23, 128logical data design 3Logical Page List 110LPL 110

Mmaintenance 2map page 61mass delete 65–66mass delete statements 64materialization 9, 120

DELETE and UPDATE 122INSERT 122SELECT 121

misusing LOBs 3


MMDBSYS 78MVS considerations 3

NNOCOPYPEND 95normalizing data 136NULL 20

Oobject orientation 6occurrence of a string 57ORGRATIO 100–101OS/390 Conversion Services 16

Ppadding 15page size 25, 153pageset structure 61partitioned base table 18partitioning 22partitions 18POSSTR 9, 45PQ42864 100, 152PQ46137 105PQ49173 152PQ51347 153PQ51847 152PQ53571 153PQ54511 76PQ59820 95, 153PRIQTY 26

RRBDP 113read performance 130rebuild pending status 113REBUILD-pending 105RECLENGTH 93recommendations for LOBs 136record identifier 20RECOVER-pending 105Redbooks Web site 162

Contact us xixREDO logging 136referring to a block of data inside of a CLO 51REPORT TABLESPACESET 99REPORT utility 22RID 20ROWID 8, 11, 18, 20, 68

adding the column 30assigning 20GENERATED BY DEFAULT 30

SSECQTY 26SELECT SUBSTR 56-SET SYSPARM 124

SHRLEVEL 98singleton 23singleton delete 65S-LOB 25S-LOB lock 59space map pages 24spanning pages 19SQL accounting 131SQL functions with LOBs 44SQLCODE -171 11SQLCODE -355 23SQLCODE -423 49SQLCODE -747 21SQLCODE -767 29SQLCODE -803 31SQLCODE -904 24, 27, 125STAR 136STATUS 93

values X and I 93store an image 80STORES 28striping 153SUBSTR 9, 45, 130SUBSTR(LOB,1,200) 129summary of considerations 2SYSIBM.SYSAUXRELS 92–93SYSIBM.SYSCOLUMNS 92SYSIBM.SYSJARCLASS_SOURCE 94SYSIBM.SYSJARDATA 94SYSIBM.SYSLOBSTATS 93SYSIBM.SYSSTRINGS 15SYSIBM.SYSTABLEPART 92–93SYSIBM.SYSTABLES 93SYSTABLEPART 93

Ttable space

partition sizes 27table space scans 19TABLESTATUS 93

values L and P 93TBNAME 92text extenders 6traces 135triggers 6TRUNCATE 97TS 27

UUCS-2 16UCS-4 16UDFs 6UDTs 6undo log records 24UNICODE 16UNICODE 367 147UNICODE support 15, 147UNIX Systems Services 83UNLOAD 96

Index 165

unload parts of LOB using locators 57unloading a LOB 54unloading a LOB using a host variable 54unloading an entire LOB using locators 55UPDATE 24, 132updating a specific part of a LOB 53UQ52836 84UQ54223 152UQ63855 76, 152UQ64118 153UQ64357 153UQ64672 152UQ64673 105, 152UQ65856 153UQ65857 153user defined functions 6user defined types 6USS 76UTF-16 16UTF-8 16

VVALUE 45VARCHAR 6VARGRAPHIC 6VDWQT 125version number 28vertical deferred write threshold 125video extender 75

WWLM Application Environment 83write performance 132

XX-LOB 25X-LOB lock 59, 62XML 7

UDFs 89UDTs 89

XML and LOBs 84XML collection 88XML COLUMN 84, 86XML Extender 81XML EXTENDER UDFs 82XML security 89XML Toolkit 83XML UDFs 81


(0.2”spine)0.17”<->

0.473”90<->

249 pages



®

SG24-6571-00 ISBN 0738425397

INTERNATIONAL TECHNICALSUPPORTORGANIZATION

BUILDING TECHNICAL INFORMATION BASED ON PRACTICAL EXPERIENCE

IBM Redbooks are developed by the IBM International Technical Support Organization. Experts from IBM, Customers and Partners from around the world create timely technical information based on realistic scenarios. Specific recommendations are provided to help you implement IT solutions more effectively in your environment.

For more information:ibm.com/redbooks

Large Objectswith DB2 for z/OS and OS/390

Define LOBs, see how they work, and see how to store them

Manage LOBs in operational environments

Use LOBs in applications and DB2 Extenders

The amount of data to be managed by workstations, middleware systems and mainframes, is growing day by day. The times are changing, and so the requirements for a database management system (DBMS) are changing.

With the incorporation of object-orientation into DB2 for OS/390, you can define new data types and functions. Some of the data objects you want to model may well be very large and complex. The foundation of object-relational extension, introduced with DB2 for OS/390 Version 6 has two major functions. One is based on support for large objects (LOB), and the other is support for user defined functions, user defined distinct types, and triggers. These large objects can contain text documents, images or even movies, and can be stored directly in the DBMS with sizes up to 2 gigabytes per object and 4.000 terabytes per LOB column.

Normally, large objects are used and manipulated through Graphical User Interfaces from a workstation. So with the implementation of LOBs we can exploit the functionalities of the DB2 family and the capacity that DB2 for z/OS provides today. The introduction of these new data types implies some changes in the administration processes and programming techniques.

In this IBM Redbook we describe the new data types, and provide some useful information on how to design and implement LOBs. We also offer examples of their use, programming considerations, and the new processes which are necessary for administration and maintenance.

Back cover


