db2 catalog & utilities

1

DB2 - IBM’s Relational DBMS - Day 4

2

Sessions Outline• Day 1:

– Session 1 : Introduction to Databases

– Session 2 : Introduction to Relational Model - DB2

• Day 2:

– Session 3 : Data Management in DB2

– Session 4 : DB2-Security

– Session 5 : SQL – Data Control Language

• Day 3:

– Session 6 : SQL – Data Definition Language

– Session 7 : SQL – Data Manipulation Language

3

Sessions Outline (Cont’d)

• Day 4:

– Session 8 : DB2- Catalog and directory

– Session 9 : DB2 – Utilities

– Session 10 : DB2 Function and Components

• Day 5:

– Session 11 : DB2 - Application Programming

– Session 12 : DB2- CICS Interface

– Session 13 : DB2- Program Preparation

– Session 14 : DB2 - QMF and SPUFI

4

Session 8

DB2- Catalog and directory

5

Session 8 - Coverage

• DB2 Catalog & Directory

• Optimizer

• Performance tuning

6

Catalog Tables & the DB2 directory

• Repository for all DB2 objects - contains 43 tables

• Each table maintains data about an aspect of the DB2 environment

• The data refers to information about table spaces, tables, indexes, privileges, on utilities run on DB2 and so on e.g. : SYSIBM.SYSTABLES, SYSINDEXES/SYSCOLUMNS ......’

7

Catalog Tables & the DB2 directory (contd...)

• When standard DB2 SQL is used, the DB2 catalog is either accessed or updated. e.g.. When a ‘CREATE TABLE’ statement is issued the catalog tables SYSIBM.SYSTABLES, SYSIBM.SYSCOLUMNS & SYSIBM.SYSFIELDS are updated.

• However the DB2 catalog is semi active only. This is because updates to number of rows, the physical order of the rows for a set of keys and the like are updated only after running a RUNSTATS utility

• DB2 catalog is integrated - DB2 catalog and DB2 DBMS are inherently bound together

8

Catalog Tables & the DB2 directory (contd...)

• It is nonsubvertible - DB2 catalog cannot be updated behind DB2’s back. i.e. if a table of 10 columns is created, it is not possible to go and change the number of columns directly on the catalog to 15. It has to be done using the standard SQL statements for dropping and recreating the table

9

DB2 Optimizer

• Analyzes the SQL statements and determines the most efficient way to access data - gives Physical data independence

• It evaluates the following factors : CPU cost, I/O cost, DB2 catalog statistics & the SQL statement

• It estimates CPU time, cost involved in applying predicates, traversing pages and sorting

10

DB2 Optimizer (contd...)

• It estimates the cost of physically retrieving and writing the data

• The information pertaining to the state of the tables that will be accessed by the SQL statements are provided by the Catalog

11

Performance Tuning

• The performance of an application can be monitored and enhanced in the application, as well as at the database level

• In application side the SQL’s can be tuned to make them more efficient, and avoid redundancy

• It is better to structure the SQL’s so that they perform only the necessary operations

12

Performance Tuning (contd...)

• On the database side, the major enhancements can be done to the definitions of tables, indexes & the distribution of tablespace and indexspace.

• The application run statistics are obtained from EXPLAIN or DB2PM (DB2 Performance Monitor) report

13DB2-DM-1

Session 9

DB2 – Utilities

14

Session 9 - Coverage

DB2 System administration

• DB2 UTILITIES

– Check

– Copy/ Mergecopy

– Recover

– Load

– Reorg

– Runstats

– Explain

15

Check

• Checks the integrity of DB2 data structures

• Checks the referential integrity between two tables and also checks DB2 indexes for consistency

• Can delete invalid rows and copies them to a exception table

• Use CHECK DATA when loading a table without specifying the ‘ENFORCE CONSTRAINTS’ option or after the partial recovery of tablespaces in a referential set

16

Mergecopy

• The MERGECOPY utility combines multiple incremental image copy data sets into a new full or incremental image copy data set

17

Recover

• Restore DB2 tablespaces and indexes to a specific instance

• Data can be recovered for single page, pages that contain I/O errors, a single partition or an entire tablespace

• Indexes are always recovered from the actual table data, not from image copy and log data, as in the case of tablespace recovery

• Standard unit of recovery is a Tablespace

18

Explain

• This feature can be used to obtain the details about the access paths chosen by the DB2 optimizer for SQL statements.

• Used specifically for performance monitoring.

• When EXPLAIN is requested the access paths that the DB2 chooses are put in coded format into the table PLAN_TABLE, which is created in the default database.

19

Explain (contd...)

• To EXPLAIN a single SQL statement precede that SQL statement with the EXPLAIN Command

EXPLAIN ALL SET QUERYNO = integer

FOR SQL statement

• The other method is specifying EXPLAIN YES with the Bind command

• Then PLAN_TABLE is to be queried to get the required information.

20

Explain (contd...)

• The information provided include the type of access of particular tables used in the SQL or Package or Plan, the order in which the tables or joined in a JOIN, whether SORT is required and so on

• Since the EXPLAIN results are dependent on the DB catalog, it is better to run RUNSTATS before running a EXPLAIN

21

LOAD Utility

• Used to load data into one or more tables

• Input:

LOAD DATA statement describing the data to be loaded

File containing data

• Output:

Loaded table or partition

Discard file of rejected records

Summary reports of errors encountered

22

LOAD Utility

• Features:

Automatic data conversion between compatible data types

Data loaded in the sequence presented; no sort invoked

Indexes built; duplicates checked

23

LOAD Utility

• Phase Flow:

UTILINIT

RELOAD

SORT

BUILD

INDEXVAL

ENFORCE

DISCARD

REPORT

24

LOAD Utility

• Example:

LOAD DATA INDDN(SYSREC)

RESUME YES

ENFORCE CONSTRAINTS

INTO TABLE TRGD55.PROJ

(PROJNO POSITION(1) CHAR(6),

PROJNAME POSITION(8) VARCHAR,

DEPTNO POSITION(33) CHAR(3))

Data File contains : 3

1 8 3

PROJ01 08training TRG

PROJ02 07Support SSG

25

REORG Utility

Reorganizes tablespace to reclaim fragmented space and improve access performance

Alter the physical space parameters for the tablespace

Unload data in a format acceptable by the LOAD utility

Reorganize tablespace or a partition

26

REORG Utility

• Input:

Name of tablespace or partition or index to be reorganized

• Output:

Reorganized tablespace and / or indexes

Table data in LOAD format

27

REORG Utility

• Phase Flow:

UTILINIT

UNLOAD

RELOAD

SORT

BUILD

UTILTERM

28

RUNSTATS Utility

Gathers information about data in tablespaces and indexes and updates DB2 catalog.

Use RUNSTATS when

– * a table is loaded

– * a table is reorganized

– * an index is created

– * extensive modifications done on tablespace

29

RUNSTATS Utility

• Input:

Tablespace for which statistics is to be updated / reported

• Output:

Report containing new statistics

Updated DB2 catalog tables

30

RUNSTATS Utility

• Phase Flow:

UTILINIT

RUNSTATS

UTILTERM

31

COPY Utility

Creates backup copies of a tablespace or a single partition

Reset copy pending status

Two types: Full image copy, Incremental image copy

32

COPY Utility

• Input:

Name of tablespace or partition to be copied

• Output:

A sequential dataset containing the imagecopy

SYSIBM.SYSCOPY updated

Copy pending status reset

33

Session 10

DB2 Function and Components

34

DB2 Catalog

Data about data DB2 is managing - “Metadata”

Stored in DB2 Tables

Can be accessed in the same way as user data

35

Transactions

SQL Command

SQL Command

SQL Command

SQL Command

COMMIT

Error Condition

ROLLBACK

Begin Transaction

Begin Transaction

End Transaction

End TransactionAll Changes “DONE”

All Changes “UNDONE”

36

Transactions : An example

Account 1

Account 1

Account 2

Account 2

Balance = 2000 Balance = 5000

Balance = 1000 Balance = 6000

TransactionWithdraw 1000 from Account 1Deposit 1000 into Account 2

37

Lost Update Problem

User 1 User2Quantity = 2000

Send 1000 units Receive 2000 unitsQuantity = 1000

Quantity = 4000

38

Incorrect Summary Problem

Read All therows twice

64104

91103

77102

80101

PercentileMarksRoll#

39

Lock on data indicates data is being used

Attribute of Locks

Object : How much data is marked “in-use”

Duration : How long is the lock needed

Mode : How severe is the lock

DB2 Concurrency - Locking Mechanism

40


• There are various classification criteria according to which DB2 locking mechanism is better understood.

– Types of Locks

– Lock Promotion / Escalation mechanism

– Size of the Locks

– Duration of Locks

– Types of Locks

• Types of Locks

– Shared Lock or ‘S’ Lock

– Exclusive Lock or ‘X’ Lock

41

• If a transaction FETCHES a record, it is imperative that it has a Shared lock on it.

• If a transaction successfully UPDATES a record, it is imperative that it has an Exclusive lock on it.

• In a FETCH / UPDATE or FETCH / DELETE sequence of operations, a Shared Lock is promoted to an Exclusive lock.

• This promotion happens through Intent locks.


42


• HOW DB2 SOLVES PROBLEMS OF CONCURRENCY

– DB2 employs LOCKING as a concurrency control mechanism.

– When a transaction needs an assurance that some database object, generally a row, will not change in some unpredictable manner while it’s back is turned, it acquires a lock on that object.

– The effect of the lock is to prevent other transactions from changing it. The transaction is thus able to carry out processing with the assumption that the object would remain stable as long as the transaction wishes it to be.

43


• Index entries, like database rows, are also subject to locking

• DB2 supports two kinds of locks, Shared and Exclusive. Other types exist but S and X locks are the most important ones. S and X locks are referred as Read and Write locks.

44


• Acquiring Locks

1. If transaction A holds X lock on row R, If transaction B requests for either S or X lock, then B would go into wait state until A releases the lock

2. If transaction A holds S lock on row R, If transaction B requests for S lock, then B would be granted the lock, If transaction B requests for X lock, then B would go into wait state until A releases the lock

45


• Acquiring Locks (Contd).

3. Transaction requests for row locks are implicit. When a transaction successfully fetches a row, it acquires an S lock. When a transaction successfully updates a row, it acquires an X lock. If a transaction holds an S lock and tries to update the row, it promotes the S lock to X lock.

4. All locks are held until the next syncpoint.

46


• Size of Locks ( LOCKSIZE Parameter )

– LOCKSIZE can be

• TABLESPACE

• TABLE

• PAGE

• SUBPAGE ( Only for Indexes )

– In general smaller the Locksize, higher is the concurrency.

– However as locksize reduces, the absolute number of locks taken at any time increases, thus using up more virtual memory and CPU.

47


• Deadlocks

– The elaborate locking mechanism solves the concurrency problems we discussed. However this introduces the possibility of Deadlocks.

– A Deadlock is a situation in which two or more transactions are in a simultaneous wait state, each waiting for the other to release a lock before it can proceed.

– If a deadlock occurs, the system detects it and breaks it. Breaking a deadlock involves choosing one of the deadlocked transactions asa victim and either rolling it back or requesting it to roll back.

– The rolled back transaction releases the locks thus allowing other transactions to proceed.

48


• Deadlocks ( Contd. )

• DB2 has a mechanism to resolve a Deadlock situation.

• When a Deadlock occurs, DB2 chooses one or more of the contending transactions as a victim and requests those transactions to release the locks for the updates of other transactions to go through.

• The victim transaction returns a negative SQLCODE.

49

DB2 Concurrency - Lock Object

PAGE or ROW

Great Concurrency

More Locking overhead

TABLE or TABLESPACE

Less Locking overhead

More data is restricted

50

DB2 Concurrency – Lock Duration

Different options for Application programs

• ACQUIRE/RELEASE

• ISOLATION LEVEL

51

DB2 Concurrency - Lock Mode

Shared Data is being Read

Other users can read the same data

Exclusive Data is being changed

No other users allowed

52

DEADLOCK

User 1 User 2

Table 1 Table 2

LockHeld

LockHeldLock Requested

53

Failures

• Hardware Failure

• Program Failure

• Natural Calamity

54DB2-FnC-15

Backing up data

• IMAGE COPY

• FULL

• INCREMENTAL

• MERGECOPY

55

System Log

[ Begin Transaction T1]

[T1, table 1, Page 1, Before Image]

[T1, table 1, Page 1, After Image]



[ Begin Transaction T2]



[ End transaction T2]

[ End transaction T1]

56

Recovery

• RECOVER Utility

Tablespace is the unit of recovery

• Point of time recovery

• Disaster Recovery

57

Integrity Enforcement

• Table Check Constraint

Designates the values that columns of a table can contain

• Referential Constraint

The limiting of a set of foreign key values to a set of primary key values.

58

Security

• Outside DB2

– RACF

• Within DB2

– Privileges

– Administrative Authorities

59

Application Programming Interface

SQL - Structured Query Language

Embedded SQL DB2 Interactive(DB2I)

Data Definition LanguageDDL

Data Manipulation LangaugeDML

Data Control LanguageDCL

SQL

60

Thank You

db2 catalog & utilities

Documents