ten essential dba tasks for db2 9 for z/os purexml guogen (gene) zhang, stsm, ibm svl,...
TRANSCRIPT
Ten Essential DBA Tasks forDB2 9 for z/OS pureXMLGuogen (Gene) Zhang, STSM, IBM SVL, [email protected] Chen, STSM, IBM SVL, [email protected]
Session Number 1810
2
Agenda
Overview of pureXML in DB2 9 for z/OS
Ten DBA tasks for pureXML in DB2 9 for z/OS
Q & A
3
What You Can Do with pureXML
Create tables with XML columns or alter table add XML columns
Insert XML data, optionally validated against schemas
Create indexes on XML data Efficiently search XML data Extract XML data Decompose XML data into
relational data or create relational view
Construct XML documents from relational and XML data
Handle XML data with the utilities and tools
XMLDOC
XML Column
XMLIndex
XML
- Managing XML data the same way as relational data
4
XML and the Web before DB2 pureXML
Client Application ServerXML
XML ObjectsMapping Mapping
Relational schema
Relational Database
XML to object mapping and object to relational mapping– Very costly– Complex– Inflexible
5
DB2 pureXML and the Web
Client
XML
DB2 pureXML
relationalXMLSOA Gateway
End-to-End XML– No expensive object mapping– Pass Thru XML from/to database
SOA-Gateway– Device/application to handle network protocols, security,
reliability, performance– Easy to manage
• Simple pre- and post-processing of XML – e.g. via XSLT
6
XML Storage on Mature Infrastructure
Base Table
XMLColDocID …
B+treeB+tree
DocID index
Internal XML Table
B+treeB+tree
NodeID index
B+treeB+tree
XML index (user)
Each XMLData column is a VARBINARY, containing a subtree or a sequence of subtrees, with context path. Rows in XML table are freely movable, linked with a NodeID index.
A table with an XML column has a DocID column, used to link from the base table to the XML table.A DocID index is used for getting to base table rows from XML indexes.
XMLDataDOCID MIN_NODEID
Regular Table space
1
2
3
1
2
2
3
02
02
0208
02
(DB2_GENERATED_DOCID_FOR_XML)
7
XML Objects for Non-partitioned Base Table
BASE Table
DOCIDINDEX
Non-Partioned Base TS
(simple, segmented, PBG)
Table for XMLCol1
NODEIDINDEX
XMLIndex
PBG TS for XMLCol1
PBG TS for XMLCol2
Cols:DOCID
MIN_NODEIDXMLDATA
Cols:DOCID
XMLCol1XMLCol2
Table for XMLCol1
NODEIDINDEX
XMLIndex
Cols:DOCID
MIN_NODEIDXMLDATA
Default: Maxpartitions – 256, SEGSIZE 4, DSSIZE 4G
8
XML Objects for Partitioned Base Table
Cols:DOCID
XMLCOL1XMLCOL2
BASE TablePart1
Partitioned Base TS2 Parts, Table has 2
XML Coumns
XMLIndex(NPI)
Partitioned TS for XMLCol1
(NPI)
Cols:DOCID
XMLCOL1XMLCOL2
BASE TablePart2
Part1 Part2
NodeIDINDEX(NPI)
Partitioned TS for XMLCol2
DOCIDINDEX
DOCIDMIN_NODEID
XMLDATA
DOCIDMIN_NODEID
XMLDATA
XMLIndex(NPI)
Part1 Part2
DOCIDMIN_NODEID
XMLDATA
DOCIDMIN_NODEID
XMLDATA
NodeIDINDEX(NPI)
DSSIZE depends on base table page size (critical: max num parts)
9
A few Things that are New related to XML
1. Implicitly created objects – DBA cannot create explicitly
2. XML indexes: XPath and keys
3. XML Schema registration
4. XMLDATA contains StringIDs in a catalog table SYSIBM.SYSXMLSTRINGS (dictionary)• UNLOAD FROMCOPY restricted, DSN1COPY
5. A new XML lock type, ID '35'x in traces• IFCID 20, 21, 107, 150, 172, and 196.
6. XML keyword in some utilities
10
Overview of utilities for XML No new special utilities for XML.
DB2 utilities support the XML data type and the related database objects
– The XML data type in LOAD and UNLOAD, with file reference support
– Support for the XML table spaces and tables used to store XML column values
– Support for auxiliary relationships used to connect base tables to XML tables
– Support for the base table space DocID index
– Support for the XML table space NodeID index and XML indexes
No partition level checking for XML PBG UTS(APAR pk49033)
DSN1COPY copied data set containing XML objects cannot be moved to another system due to StringIDs (dictionary specific to each system).
11
New Keywords or Functionality
CHECK DATA
– adds the processing of XML objects to AUXERROR and AUXONLY
LISTDEF
– implements a new XML keyword for the building of lists with and
without XML objects
LOAD
– implements a new field type, XML, in the field-spec.
– Additional keywords for XML fields to specify how white space is to
be handled (PRESERVE WHITESPACE)
UNLOAD
– implements a new field type, XML, in the field-spec
12
Ten Essential DBA Tasks for pureXML
XML-related configurations
XML document, table, and index design
Registering XML schemas
Finding dependent objects for XML
LOAD/UNLOAD
13
Ten Essential DBA Tasks for pureXML (Cont’d)
REORG and RUNSTATS
Backup and recovery
Diagnosing data problems - CHECK DATA and REPAIR
Checking query plans
Performance monitoring
14
1. XML-related Configurations
Basic XML parsing requires z/OS XMLSS: z/OS 1.8 or z/OS 1.7 with APAR OA16303
XML schemas require IBM 31-bit SDK for z/OS, Java 2 Technology Edition, V5 (5655-N98), SDK V1.5. And Java stored procedure setup.
Zparms for virtual storage: XMLVALA and XMLVALS– Default: 200MB and 10GB.– Also LOBVALA and LOBVALS impact bind-in and bind-out of
XML
Buffer pool for XML tables (default BP16K0), authorization for users who create or alter tables with XML columns.– DEFAULT BUFFER POOL FOR USER XML DATA ===>
BP16K0 BP16K0 - BP16K9
15
Command Line Processor (CLP) – Setup in Windows Official support on Unix System Service (USS), but works on
Windows Set up JCC driver on Windows Set up CLP:
– Put clp.jar in a directory, say …\path– Set up file clp.properties (optional):TerminationChar=#mydb2=hostname.svl.ibm.com:446/LOC,USER,PASSWD
– Set environment variables:• CLASSPATH = …; …\path\clp.jar• CLPPROPERTIESFILE=…\path\clp.properties
– (Bind metadata routines – execute job V91TIJMS) For convenience, on Windows:
set db2=java com.ibm.db2.clp.db2Then use %db2% to invoke CLP
16
XSR (XML Schema Repository) Setup
A set of DB2 user tables, stored procedures, and a user-defined function:– XSR tables: SYSIBM.XSROBJECTS and other XSR* tables– Schema registration: XSR_REGISTER,
XSR_ADDSCHEMADOC, XSR_COMPLETE, XSR_REMOVE– Schema validation: DSN_XMLVALIDATE (UDF)– Decomposition: XDBDECOMPXML
Setup required:– Bind package SYSXSR (install/migration jobs DSNTIJSG and
DSNTIJNX, or manual run for PTF)– Java 2 SDK V1.5, JCC for DB2 9 for z/OS, and bind the JCC
packages to SYSXSR collection ID– WLM for stored procedures and functions (default WLMENV3)– WLM for Java stored procedure (needed for XSR_COMPLETE)
17
2. XML document, table and index design
Documents should match most frequent access unit. Most efficient way of table design is hybrid.
– Put the keys in relational columns.– Use XML for its flexibility to avoid relational normalization
Use XMLTable view to experiment the design alternatives.– Same table view, different storage for performance testing
Create indexes on searched elements/attributes of XML. Use larger segment size for XML table spaces (inherited from
base for the time being) for heavy insertion.– To alleviate the contention at the end of XML TS.
18
Table Space Size Consideration
Basic XML storage is about 0.3 (strip ws w/ compression) to 1.5 (preserve ws w/o compression) of original doc size
An XML table space always use 16KB pages.– For non partitioned base table spaces, PBG table space is used
for XML.
Range-partitioned base table spaces: XML partitioning follows base table partitioning.
The number of rows to fit into a relational partition is limited by the number of documents to fit into an XML partition.– For example, 4K doc size, 32GB partition can roughly store 8M
documents (or 6M to be safe).
19
XML Indexing
Each index adds 15-20% CPU time to the basic INSERT cost. Create indexes that are only needed.– Exception: large number of XML indexes for the
benefit of flexibility Specify full path for index XML patterns, avoid
wild card, or descendant axis Rebuild index is recommended over create
index for existing data Code XPath conditions that will match index
patterns in queries.
20
Query Predicate Must Match Index Data Type<customerinfo cid=“1004”> <name>Matt Foreman</name> <phone>905-555-4789</phone></customerinfo>
Index definitionPredicate
…using xmlpattern '/customerinfo/@cid' as SQL VARCHAR(10);
…using xmlpattern '/customerinfo/@cid' as SQL DECFLOAT;
XMLEXISTS(‘$i/customerinfo[@cid = “1004”]’ …)
XMLEXISTS(‘$i/customerinfo[@cid = 1004]’ …)
It’s a numbe
r
It’s a string
Index used Index not used
What about XMLEXISTS(‘$i/customerinfo[@cid = “1004”]/name’ …) ? And XMLEXISTS(‘$i/customerinfo[@cid = 1004]/name’ …) ?
21
XML Index should “contain” query predicates
Option 1: create index idx1 on customer(info) generate key using
xmlpattern '/customerinfo/phone' as SQL varchar(40);
Option 2: create index idx2 on customer(info) generate key using xmlpattern '//phone' as SQL varchar(40);
Option 3: create index idx1 on customer(info) generate key using xmlpattern '/customerinfo/*' as SQL varchar(40);
Assume we expect predicates on “phone”.There are several ways to index this, including:
create table customer(info XML);<customerinfo cid=“1004”> <name>Matt Foreman</name> <phone>905-555-4789</phone></customerinfo>
22
Query predicate examples
Consider these index/predicate pairs:
Index definitionPredicate
…using xmlpattern '/customerinfo/phone‘
as sql varchar(35);
…using xmlpattern '//phone‘
as sql varchar(35);
…using xmlpattern '/customerinfo/*‘
as sql varchar(35);
EXISTS(‘$i//phone [. = “905-555-4789”]’ …)
XMLEXISTS( ‘$i/customerinfo[phone = “905-555-4789” ]’ …)
EXISTS(‘$i/customerinfo [* = “905-555-4789”]’ … )
<customerinfo cid=“1004”> <name>Matt Foreman</name> <phone>905-555-4789</phone></customerinfo>
23
Use Fully Specified Paths in XPath If customerinfo were a record in a programming
language, what would you do to address “city”?– customerinfo.addr.city
This is what XPath should be:/customerinfo/addr/city, not //city
Wildcard * and // should be used with a real reason.– * for generic search need– // for recursive documents
or nodes appearing at different paths or levels.
<customerinfo Cid="1004"> <name>Matt Foreman</name> <addr country="Canada"> <street>1596 Baseline</street> <city>Toronto</city> <state>Ontario</state> <pcode>M3Z-5H9</pcode> </addr> <phone type="work">905-555-4789</phone> <phone type="home">416-555-3376</phone></customerinfo>
<customerinfo Cid="1004"> <name>Matt Foreman</name> <addr country="Canada"> <street>1596 Baseline</street> <city>Toronto</city> <state>Ontario</state> <pcode>M3Z-5H9</pcode> </addr> <phone type="work">905-555-4789</phone> <phone type="home">416-555-3376</phone></customerinfo>
24
3. Registering XML Schemas
Use CLP or Data Studio for non programming interface Register schema in CLP REGISTER XMLSCHEMA
http://www.test.com/order.xsdFROM file://C:/xmlschema/order.xsdAS SYSXSR.ORDERSCHEMAADD http://www.test.com/lineitem.xsdFROM file://C:/xmlschema/lineitem.xsdADD http://www.test.com/parts.xsdFROM file://C:/xmlschema/parts.xsdCOMPLETE [ENABLE DECOMPOSITION];
Remove schema REMOVE XMLSCHEMA SYSXSR.ORDERSCHEMA;
SQL Identifier SchemaLocation
SchemaLocation
25
Enforcing Schema Validation for INSERT
Create SQL PL stored procedureCREATE PROCEDURE INSERTMYTABLEVALID(IN col1 INT, IN xmlcol BLOB(1M))LANGUAGE SQLBEGIN INSERT INTO MYTABLE VALUES( col1, XMLPARSE(DOCUMENT DSN_XMLVALIDATE(xmlcol, ‘SYSXSR.ORDERSCHEMA’)) );END
Applications callCALL INSERTMYTABLEVALID(1, :blob);
26
4. Finding Dependent Objects for XML
LISTDEF: List of related objects for utilities
REPORT TABLESPACESET
Query catalog tables:–SYSIBM.SYSXMLRELS: table.XMLcol =>internal
XMLTABLE
27
LISTDEF – define a list of related objects
Example:
LISTDEF listname INCLUDE TABLESPACES TABLESPACE tsname [RI] [ALL | BASE | LOB | XML]
Valid specifications:– BASE (non-LOB and non-XML objects)– LOB (LOB objects)– XML (XML objects)– ALL (BASE, LOB, and XML objects)– TABLESPACES (related table spaces)– INDEXSPACES (related index spaces)– RI (related by referential constraints, including informational
referential constraints)– Initial object can be: DB, TS, IS, Table, Index
28
REPORT TABLESPACESET - OutputContains both a LOB column and an XML column: POTABLE(I INT, CHARPO CLOB, PO XML). (New XML text is shown in red)
TABLESPACE SET REPORT: TABLESPACE : DBMYPO.TSMYPO TABLE : ADMF001.POTABLE INDEXSPACE : DBMYPO.IRDOCIDP INDEX : ADMF001.I_DOCIDPOTABLE
LOB TABLESPACE SET REPORT:TABLESPACE : DBMYPO.TSMYPO BASE TABLE : ADMF001.POTABLE COLUMN : CHARPO LOB TABLESPACE : DBMYPO.TSLOBPO AUX TABLE : ADMF001.TBLOBPO AUX INDEXSPACE : DBMYPO.IXLOBPO AUX INDEX : ADMF001.IXLOBPO
XML TABLESPACE SET REPORT:
TABLESPACE : DBMYPO.TSMYPO BASE TABLE : ADMF001.POTABLE COLUMN : PO XML TABLESPACE : DBMYPO.XPOT0000 XML TABLE : ADMF001.XPOTABLE XML NODEID INDEXSPACE: DBMYPO.IRNODEID XML NODEID INDEX : ADMF001.I_NODEIDXPOTABLE XML INDEXSPACE : DBMYPO.XMLIDX XML INDEX : ADMF001.XMLIDX
TABLESPACE : DBMYPO.TSMYPO BASE TABLE : ADMF001.POTABLE COLUMN : PO XML TABLESPACE : DBMYPO.XPOT0000 XML TABLE : ADMF001.XPOTABLE XML NODEID INDEXSPACE: DBMYPO.IRNODEID XML NODEID INDEX : ADMF001.I_NODEIDXPOTABLE XML INDEXSPACE : DBMYPO.XMLIDX XML INDEX : ADMF001.XMLIDX
29
Querying Catalog Tables Find internal XML table name
SELECT XMLTBOWNER, XMLTBNAMEFROM SYSIBM.SYSXMLRELSWHERE TBOWNER = ? AND TBNAME = ? AND COLNAME = ?
Find tablespace and stogroupSELECT TP.DBNAME, TP.TSNAME, TP.STORNAME, TP.PQTY, TP.SQTYFROM SYSIBM.SYSTABLEPART TP, SYSIBM.SYSTABLES TBWHERE TP.TSNAME = TB.TSNAME AND TP.DBNAME = TB.DBNAME AND TB.NAME = ? AND TB.CREATOR = ?
30
5. LOAD XML Data
LOAD uses internal INSERT for XML data – honors LOG(NO).
To load XML directly from input records, specify XML as the field type.
– LOAD DATA INDDN(INFILE) LOG NO RESUME(NO) FORMAT DELIMITED INTO TABLE PURCHASEORDERS
– LOAD DATA INDDN(INFILE) LOG NO RESUME(NO)… XMLPO POSITION(20) XML PRESERVE WHITESPACE INTO TABLE PURCHASEORDERS
To load XML from files, specify CHAR or VARCHAR along with either BLOBF, CLOBF or DBCLOBF.
Schema validation not supported for LOAD.
XML compression takes effect after first REORG, not on initial LOAD. Same for FREEPAGE, PCTFREE.
31
UNLOAD XML Data
To unload XML data directly to output records, specify XML as the field type.– non-delimited format: a 2-byte length will precede the
value of the XML.– For delimited output, no length field is present.– Limit to 32K in length
To unload XML data to separate files:– Specify CHAR(n)/VARCHAR(n) BLOBF, CLOBF or
DBCLOBF for file names– Use the template control statement to create the XML
output file and filename UNLOAD FROMCOPY is restricted
32
File References – UNLOAD to PDS (1/2)//UNLOADFP JOB ...// MSGCLASS=A,MSGLEVEL=(1,1),USER=...,// PASSWORD=...,REGION=4096K /*ROUTE PRINT ... //STEP1 EXEC DSNUPROC,UID='SAMPLE.UNLDF', // UTPROC='', // SYSTEM=‘...' //UTPRINT DD SYSOUT=* //SYSREC DD DSN=SAMPLE.UFILEREF.SYSREC,DISP=(MOD,CATLG,CATLG), // UNIT=SYSDA,SPACE=(4000,(20,20),,,ROUND) //SYSPUNCH DD DSN=SAMPLE.UFILEREF.SYSPUNCH, // DISP=(MOD,CATLG,CATLG), // UNIT=SYSDA,SPACE=(4000,(20,20),,,ROUND) //SYSUT1 DD DSN=SAMPLE.UFILEREF.STEP1.SYSUT1, // DISP=(MOD,DELETE,CATLG), // UNIT=SYSDA,SPACE=(4000,(20,20),,,ROUND) //SORTOUT DD DSN=SAMPLE.UFILEREF.STEP1.SORTOUT, // DISP=(MOD,DELETE,CATLG), // UNIT=SYSDA,SPACE=(4000,(20,20),,,ROUND) //SYSIN DD * (continued on next chart)
//UNLOADFP JOB ...// MSGCLASS=A,MSGLEVEL=(1,1),USER=...,// PASSWORD=...,REGION=4096K /*ROUTE PRINT ... //STEP1 EXEC DSNUPROC,UID='SAMPLE.UNLDF', // UTPROC='', // SYSTEM=‘...' //UTPRINT DD SYSOUT=* //SYSREC DD DSN=SAMPLE.UFILEREF.SYSREC,DISP=(MOD,CATLG,CATLG), // UNIT=SYSDA,SPACE=(4000,(20,20),,,ROUND) //SYSPUNCH DD DSN=SAMPLE.UFILEREF.SYSPUNCH, // DISP=(MOD,CATLG,CATLG), // UNIT=SYSDA,SPACE=(4000,(20,20),,,ROUND) //SYSUT1 DD DSN=SAMPLE.UFILEREF.STEP1.SYSUT1, // DISP=(MOD,DELETE,CATLG), // UNIT=SYSDA,SPACE=(4000,(20,20),,,ROUND) //SORTOUT DD DSN=SAMPLE.UFILEREF.STEP1.SORTOUT, // DISP=(MOD,DELETE,CATLG), // UNIT=SYSDA,SPACE=(4000,(20,20),,,ROUND) //SYSIN DD * (continued on next chart)
33
File References – UNLOAD to PDS (2/2)
SAMPLE.UFILEREF.SYSREC ß@ SAMPLE.XSAM0000.T054854.XSAM0000.UFILEREF(BI0MSW37) Ñ@ SAMPLE.XSAM0000.T054854.XSAM0000.UFILEREF(BI0MSW5K) `@ SAMPLE.XSAM0000.T054854.XSAM0000.UFILEREF(BI0MSW55) i@ SAMPLE.XSAM0000.T054854.XSAM0000.UFILEREF(BI0MSW6S) r@ SAMPLE.XSAM0000.T054854.XSAM0000.UFILEREF(BI0MSW7E)
SAMPLE.UFILEREF.SYSREC ß@ SAMPLE.XSAM0000.T054854.XSAM0000.UFILEREF(BI0MSW37) Ñ@ SAMPLE.XSAM0000.T054854.XSAM0000.UFILEREF(BI0MSW5K) `@ SAMPLE.XSAM0000.T054854.XSAM0000.UFILEREF(BI0MSW55) i@ SAMPLE.XSAM0000.T054854.XSAM0000.UFILEREF(BI0MSW6S) r@ SAMPLE.XSAM0000.T054854.XSAM0000.UFILEREF(BI0MSW7E)
SAMPLE.XSAM0000.T054854.XSAM0000.UFILEREF:BI0MSW37BI0MSW5KBI0MSW55BI0MSW6SBI0MSW7E
SAMPLE.XSAM0000.T054854.XSAM0000.UFILEREF:BI0MSW37BI0MSW5KBI0MSW55BI0MSW6SBI0MSW7E
TEMPLATE TCLOBF UNIT(SYSDA) DISP(MOD,CATLG,DELETE) DSN(SAMPLE.&TS..T&TI..&SN..UFILEREF) DSNTYPE(PDS) DIR(15) VOLUMES(SCR03)UNLOAD TABLESPACE SAMPLEDB.SAMPLETS PUNCHDDN SYSPUNCH UNLDDN SYSREC FROM TABLE ADMF001.SAMPLETB ( MYCOL1 POSITION(*) DECIMAL(5,2) ,MYXML1 POSITION(*) VARCHAR CLOBF TCLOBF )
TEMPLATE TCLOBF UNIT(SYSDA) DISP(MOD,CATLG,DELETE) DSN(SAMPLE.&TS..T&TI..&SN..UFILEREF) DSNTYPE(PDS) DIR(15) VOLUMES(SCR03)UNLOAD TABLESPACE SAMPLEDB.SAMPLETS PUNCHDDN SYSPUNCH UNLDDN SYSREC FROM TABLE ADMF001.SAMPLETB ( MYCOL1 POSITION(*) DECIMAL(5,2) ,MYXML1 POSITION(*) VARCHAR CLOBF TCLOBF )
&TS. Table space name&TI. Time&SN. Space nameDIR number of 256-byte records for directory
34
File References – LOAD from PDS (1/2)//LOADXML5 JOB ...// MSGCLASS=A,MSGLEVEL=(1,1),USER=...,REGION=4096K, // PASSWORD=... /*ROUTE PRINT ... //STEP1 EXEC DSNUPROC,UID='SAMPLE.LOADXML3',TIME=1440, // UTPROC='', // SYSTEM=‘...',DB2LEV=DB2A //SYSREC DD DSN=SAMPLE.UFILEREF.SYSREC,DISP=(MOD,CATLG,CATLG), // UNIT=SYSDA,SPACE=(4000,(20,20),,,ROUND) //SYSERR DD DSN=SAMPLE.LOADXML3.STEP1.SYSERR, // DISP=(MOD,DELETE,CATLG),UNIT=SYSDA, // SPACE=(4000,(20,20),,,ROUND) //SYSDISC DD DSN=SAMPLE.LOADXML3.STEP1.SYSDISC, // DISP=(MOD,DELETE,CATLG),UNIT=SYSDA, // SPACE=(4000,(20,20),,,ROUND) //SYSMAP DD DSN=SAMPLE.LOADXML3.STEP1.SYSMAP, // DISP=(MOD,DELETE,CATLG),UNIT=SYSDA, // SPACE=(4000,(20,20),,,ROUND) //SYSUT1 DD DSN=SAMPLE.LOADXML3.STEP1.SYSUT1, // DISP=(MOD,DELETE,CATLG),UNIT=SYSDA, // SPACE=(4000,(20,20),,,ROUND) //UTPRINT DD SYSOUT=* //SORTOUT DD DSN=SAMPLE.LOADXML3.STEP1.SORTOUT, // DISP=(MOD,DELETE,CATLG),UNIT=SYSDA, // SPACE=(4000,(20,20),,,ROUND) //SYSIN DD * (continued on the next chart)
//LOADXML5 JOB ...// MSGCLASS=A,MSGLEVEL=(1,1),USER=...,REGION=4096K, // PASSWORD=... /*ROUTE PRINT ... //STEP1 EXEC DSNUPROC,UID='SAMPLE.LOADXML3',TIME=1440, // UTPROC='', // SYSTEM=‘...',DB2LEV=DB2A //SYSREC DD DSN=SAMPLE.UFILEREF.SYSREC,DISP=(MOD,CATLG,CATLG), // UNIT=SYSDA,SPACE=(4000,(20,20),,,ROUND) //SYSERR DD DSN=SAMPLE.LOADXML3.STEP1.SYSERR, // DISP=(MOD,DELETE,CATLG),UNIT=SYSDA, // SPACE=(4000,(20,20),,,ROUND) //SYSDISC DD DSN=SAMPLE.LOADXML3.STEP1.SYSDISC, // DISP=(MOD,DELETE,CATLG),UNIT=SYSDA, // SPACE=(4000,(20,20),,,ROUND) //SYSMAP DD DSN=SAMPLE.LOADXML3.STEP1.SYSMAP, // DISP=(MOD,DELETE,CATLG),UNIT=SYSDA, // SPACE=(4000,(20,20),,,ROUND) //SYSUT1 DD DSN=SAMPLE.LOADXML3.STEP1.SYSUT1, // DISP=(MOD,DELETE,CATLG),UNIT=SYSDA, // SPACE=(4000,(20,20),,,ROUND) //UTPRINT DD SYSOUT=* //SORTOUT DD DSN=SAMPLE.LOADXML3.STEP1.SORTOUT, // DISP=(MOD,DELETE,CATLG),UNIT=SYSDA, // SPACE=(4000,(20,20),,,ROUND) //SYSIN DD * (continued on the next chart)
35
File References – LOAD from PDS (2/2)
LOAD DATA INDDN SYSREC LOG NO RESUME YES EBCDIC CCSID(00037,00000,00000) SORTKEYS 10 INTO TABLE "ADMF001"."SAMPLETB“ WHEN(00001:00002) = X'0003‘ ( "DSN_NULL_IND_00001" POSITION( 00003) CHAR(1) , "MYCOL1" POSITION( 00004:00006) DECIMAL PACKED NULLIF(DSN_NULL_IND_00001)=X'FF' , "DSN_NULL_IND_00002" POSITION( 00007) CHAR(1) , "MYXML1“ POSITION( 00008) VARCHAR CLOBF NULLIF(DSN_NULL_IND_00002)=X'FF' )
LOAD DATA INDDN SYSREC LOG NO RESUME YES EBCDIC CCSID(00037,00000,00000) SORTKEYS 10 INTO TABLE "ADMF001"."SAMPLETB“ WHEN(00001:00002) = X'0003‘ ( "DSN_NULL_IND_00001" POSITION( 00003) CHAR(1) , "MYCOL1" POSITION( 00004:00006) DECIMAL PACKED NULLIF(DSN_NULL_IND_00001)=X'FF' , "DSN_NULL_IND_00002" POSITION( 00007) CHAR(1) , "MYXML1“ POSITION( 00008) VARCHAR CLOBF NULLIF(DSN_NULL_IND_00002)=X'FF' )
36
File References – UNLOAD to HFS
SAMPLE.UFILEREF.SYSREC: ß@ /u/sample/clobf/BI1OFEHQ Ñ@ /u/sample/clobf/BI1OFEH0 `@ /u/sample/clobf/BI1OFEH1 i@ /u/sample/clobf/BI1OFEIN r@ /u/sample/clobf/BI1OFEIO
SAMPLE.UFILEREF.SYSREC: ß@ /u/sample/clobf/BI1OFEHQ Ñ@ /u/sample/clobf/BI1OFEH0 `@ /u/sample/clobf/BI1OFEH1 i@ /u/sample/clobf/BI1OFEIN r@ /u/sample/clobf/BI1OFEIO
$ cd /u/sample/clobf$ lsBI1OFEH0 BI1OFEH1 BI1OFEHQ BI1OFEIN BI1OFEIO
$ cd /u/sample/clobf$ lsBI1OFEH0 BI1OFEH1 BI1OFEHQ BI1OFEIN BI1OFEIO
//CRTMNT EXEC PGM=BPXBATCH, // PARM='sh mkdir /u/sample; chmod 777 /u/sample;// chown sysadm /u/sample'
//STEP1 EXEC PGM=BPXBATCH, // PARM='sh mkdir /u/sample/clobf‘...TEMPLATE TCLOBF DSN /u/sample/clobf DIR(5) DSNTYPE(HFS)UNLOAD TABLESPACE SAMPLEDB.SAMPLETS PUNCHDDN SYSPUNCH UNLDDN SYSREC FROM TABLE ADMF001.SAMPLETB ( MYCOL1 POSITION(*) DECIMAL(5,2) ,MYXML1 POSITION(*) VARCHAR CLOBF TCLOBF )
//CRTMNT EXEC PGM=BPXBATCH, // PARM='sh mkdir /u/sample; chmod 777 /u/sample;// chown sysadm /u/sample'
//STEP1 EXEC PGM=BPXBATCH, // PARM='sh mkdir /u/sample/clobf‘...TEMPLATE TCLOBF DSN /u/sample/clobf DIR(5) DSNTYPE(HFS)UNLOAD TABLESPACE SAMPLEDB.SAMPLETS PUNCHDDN SYSPUNCH UNLDDN SYSREC FROM TABLE ADMF001.SAMPLETB ( MYCOL1 POSITION(*) DECIMAL(5,2) ,MYXML1 POSITION(*) VARCHAR CLOBF TCLOBF )
37
6. Use REORG and RUNSTATS
REORG to make rows for the same documents clustered, when there are many updates and deletes.– Use real-time stats SYSIBM.SYSTABLESPACESTATS and
SYSINDEXSPACESTATS to derive heuristics (number of rows in XML TS is rows not documents)
Use RUNSTATS to collects statistics for XML data and indexes so the optimizer can pick the right access methods
LISTDEF DBACORDTSLIST INCLUDE TABLESPACES DATABASE DBACORD
RUNSTATS TABLESPACE LIST DBACORDTSLIST TABLE(ALL) INDEX(ALL)
38
7. Backup and Recovery
To recover base table space, take image copies of all related objects
– Use LISTDEF to define a list of related objects
– (QUIESCE not needed in V9)
COPYTOCOPY may be used to replicate image copies of XML objects.
MERGECOPY may be used to merge incremental copies of XML table
spaces.
Point in Time recovery (RECOVER TORBA, TOLOGPOINT)
– All related objects, including XML objects must be recovered to a consistent point in
time
Optional: CHECK utilities to validate base table spaces with XML columns,
XML indexes and related XML table spaces.
If there is an availability issue with one object in the related set, availability of
the others may be impacted.
39
8. Diagnosing Problem Related to XML Objects
Identify XML tables and their related objects – Run REPORT TABLESPACESET or query catalog tables
Validate that the auxiliary index is consistent with the underlying
table spaces– Run CHECK INDEX on all indexes, DocID, NodeID and XML value indexes
Validate the logical connection between the base table and XML
table.– Run CHECK DATA against the base table space.
Use Repair to diagnose problem related to base table spaces with
XML columns and their DocID index– Use REPAIR LOCATE KEY to locate a row using DocID key in the DocID index
Use Repair to diagnose problem related to XML table spaces and
their NodeID index or XML Value Index– Use REPAIR LOCATE RID to locate a row using a RID.
40
Checking data integrity
XML table space
CHECK INDEX on DOCID, NODEID, XML indexesCHECK DATA on base tablespace
ƒ SCOPE AUXONLYƒ AUXERROR REPORTƒ AUXERROR INVALIDATE
CHECK INDEX(2)
NODEIDIndex
Base table space
XML table space
Cols:DOCID
NODEIDXML Record Value
XMLIndex
DOCIDIndex
2 - CHECK INDEX
3 - CHECK DATA
1 - CHECK INDEX
Cols:DOCID
XMLCOL
41
Correcting XML Data
42
9. Checking query plan
CREATE TABLE ACORD.REQUEST ( ID BIGINT NOT NULL PRIMARY KEY, REQUESTXML XML, RESPONSEXML XML) IN DATABASE DBACORD
CREATE INDEX ACORD.ACORDINDEX1 ON ACORD.REQUEST(REQUESTXML)
GENERATE KEYS USING XMLPATTERN'declare default element namespace "http://ACORD.org/Standards/Life/2"; /TXLife/TXLifeRequest/TransRefGUID' as SQL VARCHAR(24)
CREATE INDEX ACORD.ACORDINDEX2 ON ACORD.REQUEST(REQUESTXML)
GENERATE KEYS USING XMLPATTERN'declare default element namespace "http://ACORD.org/Standards/Life/2"; /TXLife/TXLifeRequest/OLifE/Holding/Policy/@id' AS SQL VARCHAR(9)
43
Query plan (cont’ed)
+-------------------------------------------------------------------------------------+ | PLANNO | ACCESSTYPE | MATCHCOLS | ACCESSCREATOR | ACCESSNAME | MIXOPSEQ | +-------------------------------------------------------------------------------------+1_| 1 | M | 0 | | | 0 |2_| 1 | DX | 1 | ACORD | ACORDINDEX2 | 1 |3_| 1 | DX | 1 | ACORD | ACORDINDEX1 | 2 |4_| 1 | DI | 0 | | | 3 | +-------------------------------------------------------------------------------------+
Find participant informationabout a policy.
EXPLAIN PLAN SET QUERYNO = 101 FORSELECT XMLQuery('declare default element namespace "http://ACORD.org/Standards/Life/2"; /TXLife/TXLifeRequest/OLifE/Holding/Policy/Life/Coverage/LifeParticipant' PASSING R.REQUESTXML), XMLQuery('declare default element namespace "http://ACORD.org/Standards/Life/2"; /TXLife/TXLifeRequest/OLifE/Party [@id = /TXLife/TXLifeRequest/OLifE/ Holding/Policy/Life/Coverage/ LifeParticipant/@PartyID ] ' PASSING R.REQUESTXML)FROM ACORD.REQUEST R
WHERE XMLExists('declare default element namespace "http://ACORD.org/Standards/Life/2"; /TXLife/TXLifeRequest[TransRefGUID="2004-1217-141016-000012"]/ OLifE[Holding/Policy/@id="POLICY12"]' PASSING R.REQUESTXML)
44
Steps in a picture
XML IDX1XML IDX1 XML IDX2XML IDX2
DOCID list 1 DOCID list 2
INTERSECT
DOCID list
DOCID IDXDOCID IDX
RID list
Base Table NODEID IDXNODEID IDX1 2
34
56
Base Table
XMLColDocID …
B+treeB+tree
DocID index
Internal XML Table
B+treeB+tree
NodeID index
B+treeB+tree
XML index (user)
XMLDataDOCID MIN_NODEID
1
2
3
1
2
2
3
02
02
0208
02
Base Table
XMLColDocID …
B+treeB+tree
DocID index
Internal XML Table
B+treeB+tree
NodeID index
B+treeB+tree
XML index (user)
XMLDataDOCID MIN_NODEID
1
2
3
1
2
2
3
02
02
0208
02
45
10. Performance Monitoring and Tuning
Since XML native storage is built on top of regular tablespace structure, there are no special changes in DB2 Performance Expert to support XML other than minor points - such as new XML locks.
XML performance problem can be analyzed through accounting traces and performance traces.
XML indexes have the same consideration as other indexes.
The REORG utility should be used to maintain order and free space, if there are significant updates.
Run RUNSTATS for statistics to help pick XML indexes.
47
Other DB2 pureXML Sessions at IOD
Introduction to DB2 pureXML:TAD-1485 Wed 2.00 Querying XML Data: An Introduction for SQL Developers TLU-1712 Thu 8.30 DB2 pureXML Introduction and Survival Guide TLU-1504 Thu 11.30 Making the Transition to DB2 pureXML
DB2 pureXML Customers:TLU-1197 Mon 3.45 DB2 pureXML Production Experiences at UCLA TLU-1533 Tue 8.30 Why Rational chose DB2 pureXML for Jazz REST Services and SOA solutions BGV-1659 Tue 11.30 Implementing an Effective Electronic Government Solution - NY State Tax BHC-1677 Tue 11.30 Improving Health Care in China With a DB2 XML EMR SolutionBCS-1438 Tue 2.15 Learn How Verizon Streamlined their Order System TDZ-2146 Tue 2.15 Real-world Usage of pureXMLBGV-1659 Tue 3.45 Using XML for Effective Cross-agency Shared Services in Public Security TLU-1993 Wed 8.30 Flowers and Financial Services: B2B With DB2 pureXML TLU-1437 Wed 10.00 Implementing an Enterprise Order Database With DB2 pureXML at Verizon TAD-2255 Thu 8.30 DB2 and Data Studio: Building a Web Application Without the Web ApplicationTLU-1678 Thu 3.30 DB2 pureXML Customers - Trends and Successes
Meet the Expert:MTE-3185 Meet the Expert: Guogen ZhangMTE-3277 Meet the Expert: Henrik Loeser
48
Other DB2 pureXML Sessions at IOD
Sessions:TAD-1906 Mon 10.30 SOA and pureXML: The Role of DB2 in an Innovative ArchitectureTDZ-1489 Mon 10.30 Query XML Data in DB2 9 for z/OS BGV-1661 Wed 10.00 Streamline Govt Processing Through Electronic Forms and DB2 pureXML TLU-1622 Wed 2.00 Top 10 Best Practices for DB2 pureXMLTDZ-1810 Fri 10.00 Ten Essential DBA Tasks for DB2 9 for z/OS pureXML
Birds of a Feather:BOF-1633 Thu 5.30 DB2 pureXML Users - Best practices & Requirements BOF-1815 Thu 5.30 DB2 9 for z/OS pureXML Real-world Experiences
Hands-on Labs:HOL-1934 Tue 2.15 EForms Application With DB2 pureXML and Lotus Forms HOL-2584 Wed 10.00 DBA for pureXML in DB2 9 for z/OS HOL-2716 Wed 10.00 Advanced DB2 pureXML HOL-1933 Thu 10.00 pureXML Industry Applications HOL-2585 Thu 10.00 Learning SQL/XML with CLP and SPUFI HOL-1923 Thu 2.00 Demonstrating DB2 9.5 pureXML in an SOA Application Environment HOL-2583 Thu 2.00 Learning pureXML in DB2 9 for z/OS with IBM Data StudioHOL-1848 Fri 8.30 Introduction to DB2 pureXML 9.5
49
DB2 pureXML Resources
Web sitewww.ibm.com/software/data/db2/xml/
Wikiwww.ibm.com/developerworks/wikis/display/db2xml/
Forumwww.ibm.com/developerworks/forums/forum.jspa?forumID=1423/
Team Blogwww.ibm.com/developerworks/blogs/page/purexml
ChannelDB2 User Groupwww.channeldb2.ning.com/group/pureXML
50
Join the IDUG Worldwide XML Contest
Early Bird Prizes
Laptops
iPods
Nintendo Wii
and many more!
Five Contest Challenge CategoriesFive Contest Challenge Categories
1. Create a video
2. Write Xquery and SQL queries
3. Submit a database utility
4. Port an application
5. Write a pureXML application
Enter the Contest here at the Conference
Stop by the DB2 booth in the EXPO
Create a Video and Submit it onsite- video cameras are available
Take the Quick Quiz today at www.xmlchallenge.com
Largest investment in
XML community
200,000 Participants from
30 + countries
51
References on DB2 for z/OS pureXML
Introductory whitepaper: Leveraging DB2 9 for z/OS pureXML Technology (2nd Edition)http://www.ibm.com/developerworks/wikis/download/attachments/1824/Leveraging_DB29_for_zOS_whitepaper_v2.pdf
Detailed introduction presentation: ftp://ftp.software.ibm.com/software/data/db2zos/presentations/2007/misc/purexml.pdf
DB2 z/OS XML Guide: http://publib.boulder.ibm.com/infocenter/dzichelp/v2r2/topic/com.ibm.db29.doc.xml/bknxspsh.htm
IBM developerWorks DB2 XML (LUW): http://www.ibm.com/developerworks/db2/zones/xml/index.html
Google pureXML wikipureXML wiki
52
Implicitly Created Objects
Create Table option
Base table XML table DocID index
NodeID index
XML Index
IN database.tablespace
NM: given,DB: given,TS: given,ST: from DBBP: given or Deflt
NM: G,DB: Base,TS: G,ST: from BaseBP: Deflt/BP16K0
NM: G, TB: Base,IS: G,ST: from DBBP: Deflt/BP0
NM: G,TB: XML,IS: G,ST: from BaseBP: Deflt/BP0
NM: given,
TB: Base,IS: G,ST: given or DBBP: given or Deflt
IN DATABASE database
NM: given,DB: given,TS: G,ST: from DB
NM: G,DB: Base,TS: G,ST: from Base
NM: G, TB: Base,IS: G,ST: from DB
NM: G,TB: XML,IS: G,ST: from Base
NM: given,
TB: Base,IS: G,ST: given or DB
IN tablespace
NM: given,DB: DSNDB04,TS: given,ST: from TS
NM: G,DB: DSNDB04,TS: G,ST: from Base
NM: G,TB: Base,IS: G,ST: SYSDEFLT
NM: G,TB: XML,IS: G,ST: from Base
NM: given,TB: Base,IS: G,ST: given or SYSDEFLT
none NM: given,DB: G,TS: G, ST:SYSDEFLT
NM: G,DB: Base,TS: G, ST:SYSDEFLT
NM: G,TB: Base,IS: G,ST: SYSDEFLT
NM: G,TB: XML,IS: G,ST: SYSDEFLT
NM: given,TB: Base,IS: G,ST: given or SYSDEFLT
NM: Name, TS: Table Space, ST: Stogroup, BP: Buffer pool, TB: Table, IS: Index Space, G: Generated.
53
DSSIZE for XML Table Space (future PTF)
PBG XML DSSIZE = 4GB (base is simple, SEG, or PBG) PBR XML DSSIZE based on base DSSIZE and page size
Rationale: not to limit base table max number of partitions.SQL Ref
54
XML Related Locks New XML lock type, value ’35’x in instrumentation
SQL Base Page/Row Lock (Business as usual)
XML Lock XML Table space Page Lock
INSERT x page/row lock x lock, release at commit
x page lock, release at commit
UPDATE/DELETE u->x, s->x, x stays x x lock, release at commit
x page lock, release at commit
SELECT UR, CS-CDN None s lock, release at next row fetch
None
SELECT CS-CDY no workfile s page/row lock, release on next row fetch
s lock, release at next row fetch
None
SELECT CS-CDY workfile s page/row lock, release on next row fetch
s lock, release at close cursor
None
SELECT UR, CS-CDN, CS-CDY with Multirow fetch and dynamic scrolling
s page/row lock on rowset, release on next fetch
s lock, release on next fetch
None
SELECT RR, RS s page/row lock s lock, release at commit
None
Improvement under wayPK28265
page latch (and optional P-Lock)
page latch (and optional P-Lock)
55
© Copyright IBM Corporation [current year]. All rights reserved.U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.
THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY. WHILE EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS” WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. IN ADDITION, THIS INFORMATION IS BASED ON IBM’S CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM WITHOUT NOTICE. IBM SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION. NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, NOR SHALL HAVE THE EFFECT OF, CREATING ANY WARRANTIES OR REPRESENTATIONS FROM IBM (OR ITS SUPPLIERS OR LICENSORS), OR ALTERING THE TERMS AND CONDITIONS OF ANY AGREEMENT OR LICENSE GOVERNING THE USE OF IBM PRODUCTS AND/OR SOFTWARE.
IBM, the IBM logo, ibm.com, DB2, z/OS and pureXML are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol (® or ™), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at “Copyright and trademark information” at www.ibm.com/legal/copytrade.shtml
Other company, product, or service names may be trademarks or service marks of others.
Disclaimer