st. petersburg 1703 philadelphia 1682 george potemkin russia dmitri levin alphabroder dbanalys to...

St. Petersburg1703

Philadelphia1682

George PotemkinRussia

Dmitri Levinalphabroder

DBAnalys To The Rescue!

Disclaimer: The presentation is

highly technical and dull. Listener

need to know the database structure

well. This is a complex topic and it

requires a serious effort.ftp://ftp.progress-tech.ru/pub/Users/george/DBAnalys1_Boston.pptxftp://ftp.progress-tech.ru/pub/Users/george/DBAnalys2_Copenhagen.pptx

ftp://ftp.progress-tech.ru/pub/Users/george/DBAnalys_Copenhagen.pptx

ftp://ftp.progress-tech.ru/pub/Users/george/DBAnalys_Copenhagen.pptx

Contents


Theory

Optimization

Synergy

Practice

History

DBAnalys To The Rescue 2

PROGRESS Database AnalysisDatabase: %s Options: ixanalys chanalys Date: %s FREE CHAIN ANALYSIS===================%l block(s) found in the free chain. RM CHAIN ANALYSIS=================%l block(s) found in the RM chain. BLOCK ANALYSIS==============%l RM block(s) found in the database.%l.%l% of the RM block space is used.%l index block(s) found in the database.%l.%l% of the index block space is used. INDEX BLOCK SUMMARY % % of all Index Blocks Bytes Utiliz. Indexes %s %d %l %l %l.%l %l.%l

V51988

No tabanalys option

Manning Road in Billerica, Massachusetts


PROGRESS Database Analysis

Database:Options: tabanalys Date: Mon Nov 6 18:26:53 1995

BLOCK ANALYSIS (3906)==============

1 master block(s) found in the database.

RECORD BLOCK SUMMARY (6129) ---Record Size--- ---Fragments--- ScatterTable Records Bytes Min Max Mean Count Index Index -----------------------------------------------------------<329 lines of boring stuff deleted>Totals: 15021963 642199k 6 1675 43 15275513 1.0 5.3 802392 RM block(s) found in the database.86.10% of the RM block space is used. 270404 index block(s) found in the database.400571 free block(s) found in the database.5 index table block(s) found in the database.1 sequence block(s) found in the database.1473374 total blocks found in the database.

tabanalys option V7.3ANov, 9 1994

14 Oak Park in Bedford, Massachusetts DBAnalys To The Rescue 4

msg 3867: 1) PROGRESS Database Analysismsg 3868: Database: %smsg 8361: Blocksize: %imsg 14178: RecSpaceSearchDepth: %dmsg 6172: Options: %s %s %smsg 3870: Date: %s%rmsg 14179: 2) CHAIN ANALYSIS FOR AREA "%s" : %lmsg 14163: 2.1) FREE CLUSTER CHAIN ANALYSISmsg 14157: LIST OF FREE CLUSTER CHAIN BLOCKSmsg 14161: %J cluster(s) found in the free cluster chain.msg 3873: 2.2) FREE CHAIN ANALYSISmsg 3880: LIST OF FREE CHAIN BLOCKSmsg 14155: %J block(s) found in the free chain of %s object %imsg 16573: %rNumber of Object Object Partition/Tenant/Groupmsg 16574: Blocks Typemsg 3892: 2.3) RM CHAIN ANALYSISmsg 3897: LIST OF RM CHAIN BLOCKSmsg 14172: %J block(s) found in the RM chain of %s object %imsg 7269: 2.4) INDEX DELETE CHAIN ANALYSISmsg 7283: LIST OF INDEX DELETE CHAIN BLOCKSmsg 14176: %J block(s) found in the Index Delete chain of %s object %imsg 10097: 3) AREA "%s" : %d BLOCK ANALYSISmsg 3906: 4) BLOCK ANALYSIS (moved to the end at V11)


Segments of dbanalys output

msg 6129: 5) RECORD BLOCK SUMMARYmsg 14212: RECORD BLOCK SUMMARY FOR AREA "%s" : %lmsg 16667: RECORD BLOCK SUMMARY FOR SHARED TABLESmsg 16668: RECORD BLOCK SUMMARY FOR %s %s: %lmsg 16656: Summary for AREA "%s": %lmsg 3921: %J RM block(s) found in the database.msg 3925: 6) INDEX BLOCK SUMMARYmsg 14225: INDEX BLOCK SUMMARY FOR AREA "%s" : %lmsg 16657: INDEX BLOCK SUMMARY FOR SHARED OBJECTS:msg 16658: INDEX BLOCK SUMMARY FOR %s: %smsg 3923: %J index block(s) found in the database.msg 2314: 7) DATABASE SUMMARYmsg 16666: SUMMARY FOR AREA "%s": %lmsg 16659: SUMMARY FOR SHARED OBJECTS:msg 16579: 8) DATABASE BLOCK ANALYSIS:msg 3937: database analysis complete %s%r

msgs 0000-6130 before 8.2A msgs 7270-9999 8.3Amsgs 9894-10130 9.1Dmsgs 14155-14225 10.2Bmsgs 16641-17035 11.1


Segments of dbanalys output

History of changes

proutil db-name -C dbanalys [ [area] area-name ]V9.0ANov 21, 1998

V9.1AJan 21, 2000

Index Delete Chains / LOCKCHN (bk_frchn 2)

V10.0ADec 13, 2003

Free Cluster Chains / CLIST_FREECHN (bk_frchn 3)

proutil db-name -C dbanalys –Bp 64V9.0ANov 21, 1998


History of changes

Algorithm for calculation of Scatter Factor for SAT II tables has changed

V10.1A02Oct 27, 2006

Scatter for Type II is reported as a number of blocks occupied divided by the minimum number of blocks that table could occupy.

Scatter for Type I is reported as log10 of sum of distances between records in a table, where distance is calculated as the difference between the RECID values for the two records, divided by minimum distance possible.

In V11 that formula lost the meaning again when the database is multi-tenant or has partitioning.


History of changes

V10.1B02Aug 3, 2007

Full report on chains when running chanalys online.The following lists will be reported: LIST OF FREE CLUSTER CHAIN BLOCKSLIST OF FREE CHAIN BLOCKSLIST OF RM CHAIN BLOCKSLIST OF INDEX DELETE CHAIN BLOCKS

V10.1B02Aug 3, 2007

-recspacesearchdepthThe number of RM blocks to move from the front to the back of the RM chain before allocating a new cluster for data.Default is 5 * (cluster size / 10). Max value is 100.Old algorithm: No more than 3 blocks at a time will be examined and moved to the tail end.


History of changes

V10.1B03Nov 2, 2007

proutil <db> -C chanalys [ -scan ] -scan: fewer locks, less consistent

It instructs chanalys to use minimal locking. This may cause the chains report to be incomplete , but it will avoid turning on table locks. Offline chanalys does not use a locking mechanism, the -scan parameter will be ignored.

[Warning] Unable to acquire table lock on object 2. Chain analysis continuing with -scan option. (14167) LIST OF FREE CHAIN BLOCKS next dbkey free 51 52CTRL-C pressed - exit free chain analysis


Dance of the broken lines (the ixanalys width is 86 chars)

INDEX BLOCK SUMMARY FOR SHARED OBJECTS:--------------------------------------------Table Index Fields Levels Blocks Size % Util FactorPUB.theTable IntField IntField901234567890123456789012 10 1 IntField-1:3 2 215 1.6M 98.2 1.0 IntField-10:2 2 215 1.6M 98.2 1.0

History of changes

V11

10.2B06March, 2012

Database Analysis (dbanalys) output has changed.In the past customer's complained that the DBANALYS report widths were not wide enough resulting in a CR after the table name when the table name exceeded the Tablename column width.Now the tabanalys width is 110 chars.


History of changes

V11.4Jul 25, 2014

proutil db-name -C dbanalys [ [area] area-name ][ tenant tenant-name| group group-name| shared ][-csoutput [ -fieldsep sep-value][ -csfilePrefix pref-name ][ -verbose ]]

proutil sports –C dbanalys -csoutput -verbose

-csoutput -fieldsep sep-value (default is space )-csfilePrefix pref-name (default db-name)-verbose


Output files


sports.block.txtsports.ch.txtsports.ix.txtsports.tbl.txt

"Time Stamp" "DBUID" "DB name" "Area" "Area #" "Partition Type" "Tenancy" "Tenancy #" "Partition Policy" "Partition" "Partition #" "Owner" "Table" "Table #" "Records" "Size" "Min" "Max" "Mean“2015-04-06T14:59:00.000-4:00 "njVjvK5a8qfvEyE+7KM4WA" "sports" "Customer/Order Area" 8 "-" "-" 0 "-" "-" 0 "PUB" "Customer" 2 83 12377 118 222 149 83 1.0 1.02015-04-06T14:59:00.000-4:00 "njVjvK5a8qfvEyE+7KM4WA" "sports" "Customer/Order Area" 8 "-" "-" 0 "-" "-" 0 "PUB" "Order" 4 207 11730 53 61 56 207 1.0 1.0

-csfilePrefix

-fieldsep

-verbose

sports.lob.txtV11.6

Loading sports.tbl.txt in Excel


Loading sports.ix.txt in Excel


History of changes

11.6Oct 26, 2015

LOB support in PROUTIL DBANALYS

The PROUTIL database analysis utilities are enhanced to include analysis of LOB data. When LOB data is found in an area, a “LOB Summary” follows the “Record Block Summary” in the output, and LOB data is included in the combined summary. LOB data is also written to a new *lob.txt file when the -cscoutput parameter is used on the analysis command line.


Contents


Theory

Optimization

Synergy

Practice

History


Space: The Final Frontier… 20 Years AfterDisk Space Allocation in the Progress RDBMSG. Bjorklund, posted on PEG: Aug 15, 1995 Space: The Final Frontier, Engine Crew Monograph No. 2Gus Bjorklund, Aug 10, 1997http://www.fast4gl.com/downloads/monographs/space/space.html Space: the final frontier, OpenEdge RDBMS Internals Americas PUG Challenge 2012 , Westford, MAGus Bjorklund, May 9, 2012https://community.progress.com/community_groups/openedge_rdbms/w/openedgerdbms/791.space-the-final-frontier OpenEdge 10 RDBMS Advanced Storage ArchitectureRichard Banville, May, 2004https://community.progress.com/community_groups/user_conferences/w/exchange/2138.exchange-2004-breakout-sessions


Space: The Final Frontier… 20 Years After

1. Master Block 2. Index Block 3. Data Block 4. Free Block 5. Index Dbkey List Block 6. Sequence Block 7. Empty Block 8. Parameter Block (Not Used) 9. Area Block10. Object Directory Block (Not Used)

11. Extent List Block (Not Used)12. Object Block13. Control Block14. Area Object List Block15. Area Cluster Allocation Block16. Object Cluster List Block17. Object Block Allocation Block18. RM bit Map List Block19. Extent Header Block (bk_type 254)20. Reserved Block

Block Types (dbrpr)

There are 19 known block types.5 block types do not (should not) exist in database.4 block types do exist in database but they are not used.16 block types are mentioned by dbanalys before V11 and 14 block types by V11 dbanalys.


Block types in dbanalys

AREA "misc128_dat": 16 BLOCK ANALYSIS------------------------------------------------- 4629499 block(s) found in the area. Current high water mark: 4629439 56 free block(s) found in the area 4627892 record block(s) found in the area 0 index block(s) found in the area 60 empty block(s) found in the area 372 object block(s) found in the area 372 cluster list block(s) found in the area 2 object list block(s) found in the area 372 cluster map block(s) found in the area

0 row allocation block(s) found in the storage area.

DATABASE BLOCK ANALYSIS:----------------------- 66432530 RM block(s) found in the database.17906763 index block(s) found in the database.1 master block(s) found in the database.38 area block(s) found in the database.1 control block(s) found in the database.2 object block(s) found in the database.2905 cluster list block(s) found in the database.2905 cluster allocation block(s) found in the database.2905 object block(s) found in the database.42 object list block(s) found in the database.2905 object allocation block(s) found in the database.91506 free block(s) found in the database.0 index table block(s) found in the database.1 sequence block(s) found in the database.692370 empty block(s) found in the database.


Types of Storage Areas

1. Storage Area Type I

2. Storage Area Type II

3. Storage Area Type IIII + II Hybrid (V10.0B)

Blocks could be in the areas of the following types:

Used for temp-tables ( DBI file )


Data ClustersStorage Area Type II

Three possible cluster sizes: 8, 64, or 512 adjacent blocks

Clusters are allocated from the area and given to a particular object.They are maintained by a doubly linked list.

First and last blocks in the cluster have the extended headers (80 bytes instead of 64). First block stores Owner Info (transactionId, serialNumber)Last block stores Cluster Link (firstFreeCluster, lastFreeCluster) Database extend: MAX(64, cluster size) – that means that database will be extended for 1 cluster if the cluster size is 64 or 512 and 8 clusters if cluster size is 8.


Chain Types

1. Cluster chains

2. Block chains

3. F-chains


;-)

Cluster Chains1. Area Control Clusters

2. Data Object Clusters

3. Free Cluster Chains (CLIST_FREECHN)

Block Chains1. Free Chains (FREECHN, bk_frchn 0)

2. RM Chains (RMCHN, bk_frchn 1)

3. Index Delete Chains (LOCKCHN, bk_frchn 2)

F-Chains Z unofficial name ;-)

Free Cluster Chains


proutil sports –C dbrpr15. Display Cluster Chain (Type II Area)

Cluster Link of object 1/4Total Clusters: 2 First cluster: 768 Last cluster: 1536 768 1536 1536 02 cluster(s) found

Cluster Link of object 2/21Total Clusters: 1 First cluster: 1024 Last cluster: 1024 1024 01 cluster(s) found

Clusters of the“Order” table (object 1/4)and its “Cust-Order” index (object 2/21).

Fast Object DeletionArea Free Cluster LinkFirst cluster: 1024 Last cluster: 1536 1024 768 768 1536 1536 03 cluster(s) found

Free cluster chain after the “Order” tableand its “Cust-Order” index were deleted.

Free Cluster Chains

CHAIN ANALYSIS FOR AREA "Customer/Order Area" : 8---------------------------------------------------------- FREE CLUSTER CHAIN ANALYSIS--------------------------- LIST OF FREE CLUSTER CHAIN BLOCKS next dbkey free 1024 768 768 1536 1536 03 cluster(s) found in the free cluster chain.


Free Cluster Chains


Block Chainsproutil db –C chanalys [–scan]


LIST OF FREE CHAIN BLOCKS next dbkey free 7026440 70264392046268 block(s) found in the free chain of Index object 22

LIST OF INDEX DELETE CHAIN BLOCKS dbkey Block 722 24993 block(s) found in the Index Delete chain of Index object 33

LIST OF RM CHAIN BLOCKS next dbkey Block 804512704 2372 49 08534342 block(s) found in the RM chain of Table object 4

Index Delete chain


20731 block(s) found in the Index Delete chain of Index object 622

0 block(s) found in the Index Delete chain of Index object 622

proutil db-name -C idxcompact item-tax.item-group

IndexObjectBlock Free

BlockFreeBlock

FreeBlock

FreeBlock

IxdelBlock

IxdelBlock

IxdelBlock

IxdelBlock

(Unique Indexes only)

Contents


Theory

Optimization

Synergy

Practice

History


Dbanalys Processing Phases

[2015/10/19@12:49:02.602-0400] CHAIN ANALYSIS FOR AREA "Order Area" : 9[2015/10/19@13:18:16.944-0400] 1402490 block(s) found in the RM chain of Table object 362

[2015/10/19@13:18:16.960-0400] AREA "Order Area" : 9 BLOCK ANALYSIS[2015/10/19@13:26:11.385-0400] 4221951 block(s) found in the area.

Scan 4 types of the chains:FREE CLUSTER CHAIN ANALYSIS

FREE CHAIN ANALYSISRM CHAIN ANALYSIS

INDEX DELETE CHAIN ANALYSIS ) Chanalys on/offline

Sequential scan of all blocks in the areaScan the Fifth type of the chains: “F-chains”

Dbanalys offline

Timing of dbanalys run

proutil dbname -C dbanalys | \while IFS="" read Linedo echo `date '+%H:%M:%S'` "$Line"done > dbname.dbanalys.log

10:19:52 INDEX DELETE CHAIN ANALYSIS10:19:52 -----------------10:19:52 AREA "Order-line Area" : 7 BLOCK ANALYSIS10:19:52 -------------------------------------------------10:19:5211:55:37 13212671 block(s) found in the area.11:55:3711:55:37 Current high water mark: 1321267111:55:37


Timing of dbanalys run

PROMSGS=$DLC/promsgs.dbaexport PROMSGSDB=dbnameOffset=`ls -l $DB.lg | awk '{print $5}’`(proutil $DB -C dbanalys && tail +${Offset}c $DB.lg) >$DB.dbanalys.log

Use ModPromsgs.p to create the promsgs.dba file:ftp://ftp.progress-tech.ru/pub/Users/george/Programs/ModPromsgs.p


[2015/10/06@17:59:12.425-0400] P-10944600 T-1 I : (-----) AREA "Order Area" : 9 BLOCK ANALYSIS[2015/10/06@18:01:20.312-0400] P-10944600 T-1 I : (-----) 2997247 block(s) found in the area.127.887 sec = 00:02:07.887

ftp://ftp.progress-tech.ru/pub/Users/george/Programs/ModPromsgs.p

Free Cluster Chain Scan Time

[2015/09/29@13:13:48.269-0400] P-37552166 T-1 I Usr 19: (-----) CHAIN ANALYSIS FOR AREA "Misc1 Area" : 21[2015/09/29@13:13:48.269-0400] P-37552166 T-1 I Usr 19: (-----) FREE CLUSTER CHAIN ANALYSIS[2015/09/29@13:17:24.353-0400] P-37552166 T-1 I Usr 19: (-----) 56325 cluster(s) found in the free cluster chain.

[2015/09/29@14:55:51.512-0400] P-37552166 T-1 I Usr 19: (-----) CHAIN ANALYSIS FOR AREA "ic-ledger index Area" : 26[2015/09/29@14:55:51.512-0400] P-37552166 T-1 I Usr 19: (-----) FREE CLUSTER CHAIN ANALYSIS[2015/09/29@14:56:26.651-0400] P-37552166 T-1 I Usr 19: (-----) 5726 cluster(s) found in the free cluster chain.

[2015/09/29@17:06:05.698-0400] P-37552166 T-1 I Usr 19: (-----) CHAIN ANALYSIS FOR AREA "item-ledger Index Area" : 38[2015/09/29@17:06:05.698-0400] P-37552166 T-1 I Usr 19: (-----) FREE CLUSTER CHAIN ANALYSIS[2015/09/29@17:19:24.490-0400] P-37552166 T-1 I Usr 19: (-----) 257338 cluster(s) found in the free cluster chain.


Free Chain Scan Time


RM Chains Scan Time


[2015/10/19@10:25:01.825-0400] P-14155958 T-1 I Usr 19: (-----) RM CHAIN ANALYSIS[2015/10/19@11:43:01.389-0400] P-14155958 T-1 I Usr 19: (-----) 3755255 block(s) found in the RM chain of Table object 370[2015/10/19@11:56:02.210-0400] P-27394110 T-1 I Usr 19: (-----) RM CHAIN ANALYSIS[2015/10/19@12:39:30.437-0400] P-27394110 T-1 I Usr 19: (-----) 2136418 block(s) found in the RM chain of Table object 63[2015/10/19@12:49:03.141-0400] P-11010220 T-1 I Usr 19: (-----) RM CHAIN ANALYSIS[2015/10/19@13:18:16.944-0400] P-11010220 T-1 I Usr 19: (-----) 1402490 block(s) found in the RM chain of Table object 362[2015/10/19@13:26:12.032-0400] P-7995430 T-1 I Usr 19: (-----) RM CHAIN ANALYSIS[2015/10/19@13:35:16.730-0400] P-7995430 T-1 I Usr 19: (-----) 185946 block(s) found in the RM chain of Table object 520[2015/10/19@13:38:06.351-0400] P-37028030 T-1 I Usr 19: (-----) RM CHAIN ANALYSIS[2015/10/19@13:44:57.849-0400] P-37028030 T-1 I Usr 19: (-----) 177890 block(s) found in the RM chain of Table object 267[2015/10/19@13:45:48.519-0400] P-31916152 T-1 I Usr 19: (-----) RM CHAIN ANALYSIS[2015/10/19@13:53:49.885-0400] P-31916152 T-1 I Usr 19: (-----) 150138 block(s) found in the RM chain of Table object 287

AREA BLOCK ANALYSIS Scan Time


“F-chains” = the fragmented records

proutil db-name –C ixanalysproutil db-name –C tabanalys

Table Fragmentation Affects the Reading

Block’s DBKEY Type Chain Backup Ctr

Next DBKEY in Chain Block Update Counter

Free SpaceFreeDirs. Rec 0 Offset Rec 1 Offset

Rec 2 Offset Rec n Offset

NumDirs.

Free Space

Used Data Space

Record 0

Record 1

Record 1





NumDirs.

Free Space

Used Data Space

Record 0

Record 1

Record 1

Language Record Buffer (used by 4GL clients but not by dbanalys)

Rowid 9794

Rowid 2828





NumDirs.

Free Space

Used Data Space

Record 0

Record 1

Record 1

Rowid 10240

Connect All Fragments

DB-08: A Day in the Life of a Type II RecordRichard BanvilleExchange 2005


Contents


Theory

Optimization

Synergy

Practice

History


Record Packing Factor

The minimum amount of free space needed in a block for it to remain on the RM Chain.Default: 300 bytes (150 bytes for 1K block size)Range: from 1 (from Create Limit) to (blocksize – 128)

The minimum amount of free space required in a block to allow creation of a new record starting in the block. Default: 150 bytes (75 bytes for 1K block size)Range: from 32 to (blocksize – 128)

The number of RM blocks to move from the front to the back of the RM chain before allocating a new cluster for data.Default: 5 (clustersize / 10)Range: from 0 to 100

Create Limit

-recspacesearchdepth

RPB One value per area is set in .st file. It is powers of 2 from 1 to 256.


Toss Limit

Create/Toss limitsproutil db -C dispTos[sCreateLimits] area*proutil db -C setAreaCr[eateLimit] area* limitproutil db -C setBlobCr[eateLimit] lob* limit (deprecated in V11)proutil db -C setLobCre[ateLimit] lob* limitproutil db -C setTableCr[eateLimit] table limit proutil db -C setAreaTo[ssLimit] area* limitproutil db -C setBlobTo[ssLimit] lob* limit (deprecated in V11)proutil db -C setLobTos[sLimit] lob* limitproutil db -C setTableTo[ssLimit] table limit [ tenant tenant-name| group group-name| shared ] * - For an area in V11 you can specify either the name or the number. And in V10 only the area number could be specified. And the same for LOBs. For a table only the name could be specified in any version.


Proutil updates: _StorageObject._Create-Limit_StorageObject._Toss-Limit

Toss Limit

proutil db -C chanalys “Order Area” -scan

RM CHAIN ANALYSIS-----------------

LIST OF RM CHAIN BLOCKS free # free dbkey space slots hold 58719520 1337 5 0 58719536 8124 16 0 58719552 8124 16 0...

58718960 682 5 0 58719456 669 5 0

1313186 block(s) found in the RM chain of Table object 362


Number of blocks on the RM chain with free space above “X”

Toss Limit


Number of blocks on the RM chain with free space above “X”

Toss LimitWhat value to choose?


Toss LimitRule of Thumb

Set toss limit = average record length + 10-20%


Distribution of Free Space in RM Chain Inside the Area



Distribution of RM Chain Blocks Inside the Area

Chains in dbrpr


DATABASE SCAN MENU ------------------

1. Report Bad Blocks 3. Fix Bad Blocks 4. Report Bad Records 5. Delete Bad Records 6. Dump Records to RM FileON 7. Rebuild Free ChainON 8. Rebuild RM ChainON 9. Rebuild Index Delete Chain 10. Change Current Working AreaON 11. Fix Cluster Chains in Type II Area A. Apply scan to all areas

G. Go

Q. Return to Main Menu

(Current Working Area: Customer/Order Area)

DATABASE REPAIR MENU --------------------

1. Database Scan Menu 2. Test One or More Indexes 3. Remove Bad Record Fragment 4. Dump Block 5. Load Block 6. Copy Bytes Between Files 7. Load RM Dump File 8. Reformat Block to a Free Block 9. Change Current Working Area10. Display the Free Chain11. Display the RM Chain12. Display the Index Delete Chain13. Display Block Contents14. Display Record Contents15. Display Cluster Chain (Type II Area)16. Scan/Fix block checksum (Type II Area) A. Try to read additional Index/Table info P. Print Info Menu

Q. Quit

Long RM chain


proutil db-name -C dbrprdbanalys before dbrpr1313186 block(s) found in the RM chain of Table object 362

Scan Backward (Yes/No)?

Scan of Order Area extent 1 10% completeScan of Order Area extent 1 20% completeScan of Order Area extent 1 30% completeScan of Order Area extent 1 40% completeScan of Order Area extent 1 50% completeScan of Order Area extent 1 60% completeScan of Order Area extent 1 70% completeScan of Order Area extent 1 80% completeScan of Order Area extent 1 90% completeScan of Order Area extent 1 100% complete

dbanalys after dbrpr120813 block(s) found in the RM chain of Table object 362

Create Limit: For now we see through a glass, darkly

promon db-name Activity: Space Allocation USA KZ RU1 RU2 LOAD Per Sec Per Sec Per Sec Per Sec Per SecDatabase extends 0.29 4.94 7.02 4.99 120.13Take free block 0.30 1.66 1.99 1.79 33.11Return free block 0.01 0.04 0.18 0.06 0.00Alloc rm space 53.69 196.27 130.62 387.88 2083.33Alloc from rm 53.69 196.26 130.62 387.88 2070.93Alloc from free 0.00 0.00 0.00 0.00 12.40Bytes allocated 8538.09 17411.29 22496.94 38779.66 372915.41rm blocks examined 55.69 198.46 131.42 389.71 2070.93Remove from rm 2.02 4.23 12.44 12.88 99.21Add to rm, front 0.0016 0.0129 0.0105 0.0065 12.40Add to rm, back 0.47 1.11 7.14 9.66 0.00Move rm front to back 1.18 1.55 0.05 1.42 0.00Remove locked rm entry 14.54 18.50 20.60 180.48 0.00Add to rm, front (blocks) 0.82 6.59 5.37 3.34 99.21


Create Limita.k.a. Expansion Space Constant

proutil db -C setAreaCreateLimit area limitproutil db -C setTableCreateLimit table limit



Create Limit. The “Light Breathing” Space

RECORD BLOCK SUMMARY FOR AREA "item-summary Area" : 35------------------------------------------------------- -Record Size (B)- ---Fragments--- ScatterTable Records Size Min Max Mean Count Factor FactorPUB.item-summary 34301938 15.2G 419 1066 474 35430542 1.0 1.0

1128604 records (3%) are fragmented.

proutil db -C chanalys "item-summary Area" -scanCHAIN ANALYSIS FOR AREA "item-summary Area" : 35

Date: Mon Oct 19 13:45:48 2015 Date: Tue Oct 20 12:35:22 2015

150138 block(s) found in the RM chain 150460 block(s) found in the RM chain 2288127 block(s) found in the area. 2288127 block(s) found in the area.

15378 blocks were changed (10% of all blocks)15258 blocks were updated 234 blocks were removed from the chain 0 blocks were added to the chain’s head 556 blocks were added or moved to the bottom 322 blocks = chain's increase

Create Limit. The “Light Breathing” Space

proutil db -C chanalys "item-summary Area" –scanRPB 16

LIST OF RM CHAIN BLOCKS LIST OF RM CHAIN BLOCKS free free # free free free # freedbkey space1 space2 slots dbkey space1 space2 slots 36597840 8036 8035 -1 15 35558848 7678 7669 -9 14 36597872 8037 8037 0 15 35558864 6879 6882 +3 12 36597968 7705 7675 -30 12 36235728 7674 7674 0 15 36597984 8124 8124 0 16 36235744 7975 7976 +1 15 36598016 8000 8000 0 14 35558880 7436 7453 +17 13 36598064 8054 8054 0 15 36235760 7223 7223 0 12 36598160 7934 7919 -15 14 35558896 6539 6540 +1 10 36598192 8124 8124 0 16 36235776 7213 7206 -7 13 36598208 7990 7960 0 15 35558912 6597 6664 +67 10 36598336 7935 7874 -61 14 36235792 6506 6498 -8 11 36598432 8124 8124 0 16 35558928 7535 7558 +23 13 36598448 6711 6711 0 13 36235808 7247 7244 -3 14


Create Limit. The “Heavy Breathing” Space

proutil db -C tabanalys "Order Area”RECORD BLOCK SUMMARY FOR AREA "Order Area" : 9------------------------------------------------------- -Record Size (B)- ---Fragments--- ScatterTable Records Size Min Max Mean Count Factor FactorPUB.order 40840212 29.1G 450 13519 764 49388337 1.2 1.0

8548125 records (21%) are fragmented.1396614 block(s) found in the RM chain of Table object 3624174847 block(s) found in the area.33% of data blocks are on RM chain.


Chain reduced by 786 blocks (from 1,403,276 to 1,402,490).HWM increased by 4096 blocks or by 8 clusters (from 4,221,951 to 4,226,047).1329 blocks (0.1%) on RM chains were updated.9 blocks were moved to the chain’s back.22 records were deleted from blocks on RM chain.

proutil db -C chanalys "Order Area" –scanRPB 16


Create Limit. The “Heavy Breathing” Space

Create Limit. Table’s Updates vs. Creates

Updates/Creates

0 : 1 ?

1 : 1 ?

many to any ?


Create Limit. Updates Immediately After Creates

DO TRANSACTION: CREATE Customer. ASSIGN Customer.Name = “George”.END.

_TableStat-create: 1 _TableStat-update: 0 ???

DO TRANSACTION: CREATE Customer.

TRIGGER PROCEDURE FOR Create OF Customer./* Automatically Increment Customer Number using Next-Cust-Num Sequence */ASSIGN Customer.Cust-Num = NEXT-VALUE(Next-Cust-Num).

ASSIGN Customer.Name = “George”. END.

_TableStat-create: 1_TableStat-update: 1


Min Record Length


Mean Record Length


Create/Toss Limits

"Philosophy"

1. No fragmentation, no long chains –> don’t change the create and toss limits. Your efforts will be worthless.

2. First decide on the value of the create limit. Treat it as a "breathing" space for all records stored in the same block rather then an expansion of an individual record. Identify a type of record updates: frequent, only-after-create, almost-never.

3. The toss limit must be larger than the create limit. Use it to find a balance between the efficient space usage inside a database and the keeping RM chains relatively short. If it is impossible to reach the balance than just leave it and relax.

Defragmentation

© 2007 Progress Software Corporation63 DB-12: Atlantis, The Holy Grail, and 5-9’s: In Search of the Perfect World

Features That Increase Planned Availability

Online Schema Updates

Auto-Defragging of the database

Self Healing Attributes• Truncate log file online

• Adding extents online

Enabling AI Online

SQL Revoke Privileges

OpenEdge Replication Failback


AreaDefrag.p

Version 2.0 (“Precision weapon”)

1. Works online (4GL)

2. Defrags only records that are fragmented

3. Defrags one area at a time

4. Allows to specify a range of blocks

5. Could be multi-threaded

6. Could be interrupted


AreaDefrag.p

Syntax: RUN ScanArea( type, area-number, position, max-blocks)Type: TRUE – with defragmentation FALSE – report only Position: Either from block-number or to block-number Unknown value “?” Means HWM and then max-blocks should be negative


AreaDefrag.p

Examples: RUN ScanArea( FALSE, 7, 1, 100000)

scan area 7 without defragmentation from the first block forward 100 thousand blocks

RUN ScanArea( TRUE, 7, ?, -100000)

scan area 7 with defragmentation from HWM backwards 100 thousand blocks


Testing results


for each <table> no-lock use-index <small-index>:end. Before defrag.p : 2 hr 19 min or 139 minAfter defrag.p : 22 min or more then 6 times faster

proutil -C dump index 0 Before defrag.p : 102 minAfter defrag.p : 10 min or more then 10 times faster

Contents


Theory

Optimization

Synergy

Practice

History


_TableStat_IndexStat_TableStat_IndexStatDbanalys


synergy


synergy


Dbanalys

synergy


George [email protected] [email protected]


mailto:[email protected]

mailto:[email protected]

Thank you for your time!

st. petersburg 1703 philadelphia 1682 george potemkin russia dmitri levin alphabroder dbanalys to...

Documents