Download - B2: What’s New in 10.1 RDBMS?
B2: What’s New in 10.1 RDBMS?
So many features, so few releases
Richard BanvilleProgress Fellow
© 2008 Progress Software Corporation2
Agenda
High Availability• Problem Avoidability
• Visibility
• Scalability
Maintainability
The database “ilities”
© 2008 Progress Software Corporation3
Database Consistency Checking
Index orderId in order for recid 2010 could not be deleted.
Wrong key in idx 10 for record 2010.
Invalid size of an index entry.
Seen these messages before?
© 2008 Progress Software Corporation4
Database Consistency Checking
Invalid RM block for area 10
rmdoins: pbk->free went negative dbkey 4096
bkwrite: bktbl dbk 4096 not equal to bkbuf dbk -1234
bkaddr called with negative blkaddr: -1234
Or how about these…
© 2008 Progress Software Corporation7
Database Consistency Checking
Shared memory overwrite protection• -MemCheck
Physical block consistency checking• -DbCheck
• -AreaCheck “area name”
• -Index Check “index name”
• -TableCheck ‘table name”
Stop problems before they happen
© 2008 Progress Software Corporation8
Database Consistency Checking
Database startup parameter Managed via promon R&D Admin Functions
• 8. Block level consistency check
Enabling the consistency checks
Current consistency check status:
1. -MemCheck: enabled2. -DbCheck: enabled3. -AreaCheck in area: "customer" enabled4. -IndexCheck: disabled5. -TableCheck: disabled
Enter the option to enable/disable a consistency check:
© 2008 Progress Software Corporation9
Database Consistency Checking
Memory checking: unnoticeable impact Block level checking
Performance impact
Current consistency check status:
1. -MemCheck: enabled2. -DbCheck: enabled3. -AreaCheck in area: "customer" enabled4. -IndexCheck: disabled5. -TableCheck: disabled
Enter the option to enable/disable a consistency check:
~5%
< 1%
© 2008 Progress Software Corporation10
Database Consistency Checking
dbtool block consistency checking
5. Read or Validate Database Block(s)
Validation levels• 0: Block header info only• 1: Record header & record size• 2: Record overlap checking
3. Record Validation (logical validation)
Looking for existing inconsistencies online
© 2008 Progress Software Corporation11
Online Tools
Online dbanalys - Includes chanalys info Tools now online
proutil <db> -C chanalys [ -scan ]
• -scan: fewer locks, less consistent
proutil <db> -C idxcheck
• Idxcheck validation levels– Physical consistency– Keys to records– Records to keys– Validate key order– Locks associated tables
More analysis available online
contention
© 2008 Progress Software Corporation12
The roll forward process
myDb
myDb.bak
© 2008 Progress Software Corporation13
The roll forward process
myDb
ai ai ai ai
ftp
© 2008 Progress Software Corporation14
The roll forward process
myDbHot
Standbyai ai ai ai
ftp
ai ai ai ai
X
SYSTEM ERROR: Attempt to read block 18446744073709550382 which does not exist in area 8, database x.
** Save file named core for analysis by Progress Software Corporation.
Roll forward
© 2008 Progress Software Corporation15
Ai Verify
rfutil <db> -C aiverify <type>
• Partial: ai block and note header validation– Increases reliability of archived ai files
• Full: partial + note data validation– Identifies point in time recovery
• Running– At ai switch or on ai archival– Just before roll forward of extent
Preferably on hot standby
Ai validation before application
© 2008 Progress Software Corporation16
Roll forward verification
myDbHot
Standbyai ai ai ai
ftp
ai ai ai ai
X
Hot Stand by:• Validate/fix production db• Re-base hot standby
Recovery Scenario:• Roll forward to transaction
rfutil myDb -C aiverify full
rlNoteVerify: Note dbkey is negative -1234. (14099)Trid: 358 code = RL_CXINS version = 2 (12528)
© 2008 Progress Software Corporation17
More tools for high availability
“Online” backup of replication target (foundational work)• Normal operating state required
• SHR schema lock on source
• ai file stores changes until complete
EMC’s SRDF* certification• Real time copies of logical data volumes
• Data replication
• Remote backup
Replication enhancements
*Symmetrix Remote Data Facility (SRDF)
• Disaster recovery
• Fail over/fail back
© 2008 Progress Software Corporation18
The problem
Out of free shared memory. Use -Mxs to increase.
Lock table overflow, increase -L on server.
Have you ever seen these error messages…
or heard these…
Why can’t you improve the buffer pool hit ratio on the database?
The recovery subsystem is a bottle neck. Look at that BI buffer wait %.
© 2008 Progress Software Corporation19
Increase startup parameters online
proutil <db> -C increaseto <params> <params>: -L, -B, -bibufs, -aibufs, -Mxs Increase, not decrease Resource restrictions apply New shared memory segments
• Security restrictions– Servers: automatically attach quickly– Self serve: attach w/db action over time
• Segment size
Increase startup parameters without database restart
© 2008 Progress Software Corporation20
Increase startup parameters online
Increasing available locks online
proutil myDb -C increase -L 10000
Waiting for broker connection to newly added shared memory segments. Usr Name Type Pid 7 richb ABL 5957
The connections above have not attached to recently added shm segments.Do you wish to recheck? (y/n)
(n): Increase params aborted because of shared memory allocation issue.
(y): Increase params increasing lock table size (-L) from 1025 to 10016.
© 2008 Progress Software Corporation21
Agenda
High Availability• Problem Avoidability
• Visibility
• Scalability
Maintainability
The database “ilities”
© 2008 Progress Software Corporation22
Promon
Promon R&D Status• 17. Servers By Broker
• A more organized view of existing data
Better organized server grouping
Sv Pend. Cur. Max. Port No Pid Type Protocol Logins Users Users Users Num
0 15275 Login TCP 5 0 0 15 2053 2 15501 Auto TCP 1 0 1 15 1025 3 15509 Auto TCP 1 0 1 15 1026 4 15511 Auto TCP 1 0 1 15 1027 5 15514 Auto TCP 1 0 1 15 1028 1 15381 Login TCP 5 0 0 5 2051 7 15609 Auto TCP 1 0 1 5 1030 8 15617 Auto TCP 1 0 1 5 1031 9 15629 Auto TCP 1 0 1 5 1032 10 15638 Auto TCP 1 0 1 5 1033
© 2008 Progress Software Corporation23
Promon
Promon R&D Other Displays• 7. Total Locks per User
User type display – _Connect-ClientType
Improved user information
User Name Type PID TTY Total Record SHR/EXCL... 5 richb SELF/ABL 15494 /dev/pts/16 1 1 1 11 richb SELF/ABL 16101 /dev/pts/13 3 3 0 24 richb REMC/SQLC 15530 2 1 0 48 richb REMC/WTA 20182 mysystem 2 2 0100 richb REMC/ABL 20183 mysystem 2 2 0101 richb SQFC 20100 mysystem 5 4 0150 richb REMC/APSV 20101 mysystem 1 1 0175 richb SELF/APSV 20102 mysystem 1 1 0
© 2008 Progress Software Corporation24
Statement Caching
List recent client statements• Promon R&D Status
18. Client Database-Request statement Cache– By user/server/all users current and future.– Last line or entire stack– ABL info obtained from DEBUG-LIST output
.i’s are in-lined
What code is executing against my database
© 2008 Progress Software Corporation25
Statement Caching
Where is that ABL code executing
User number : 24 User name : richb User type : REMC/ABL Login date/time : 03/06/08 15:30
Statement caching type : ABL Program StackStatement caching last updated : 03/06/08 15:35
Statement cache information : 39 : proc7 /usr1/richb/x.p 26 : proc6 /usr1/richb/x.p 22 : proc5 /usr1/richb/x.p 18 : proc4 /usr1/richb/x.p 14 : proc3 /usr1/richb/x.p 10 : proc2 /usr1/richb/x.p 6 : proc1 /usr1/richb/x.p 3 : /usr1/richb/x.p
© 2008 Progress Software Corporation26
Statement Caching
What’s that SQL code executing
User number : 23 User name : richb User type : REMC/SQLCLogin date/time : 03/06/08 15:42
Statement caching type : SQL StatementStatement caching last updated : 03/06/08 15:42
Statement cache information : select count(*) from pub.customer
© 2008 Progress Software Corporation27
Statement Caching
proutil <db> -C updatevst• Need to load new schema fields
_Connect vst• _Connect-CachingType
• _Connect-CacheLastUpdate
• _Connect-CacheInfoType
• _Connect-CacheLineNumber[32]
• _Connect-CacheInfo[32]
VST support - _Connect
© 2008 Progress Software Corporation28
Agenda
High Availability• Problem Avoidability
• Visibility
• Scalability
Maintainability
The database “ilities”
© 2008 Progress Software Corporation29
Scalability
Large file support for bulk load (> 2Gb)• Independent of DB large file status
IPv6 support• More ip addresses
– only 30% ip addrs left (7 yrs)
• Routing improvements• Required by government contracts• Configuration
– -ipver IPv4 (default) or IPv6– Property file: ipver=– Explorer option
I want more
© 2008 Progress Software Corporation30
Scalability
IPv6 support• Mixed mode (dual stack)
– IPv6 can convert IPv4 address
– Not available on windows
• Avoiding confusion– -minport/-maxport
– use IPv6 configured –H hostnames
I want more
© 2008 Progress Software Corporation31
Internal Performance Improvements
“clean” shutdown indicator• Avoids long redo in roll forward
• Last 2 clusters always redone
Improved read operation concurrency• Latch enhancements and usage optimization
Multi-user bi activity optimization• Avoid rollback “block jump” notes
• Fewer notes written
The need for speed
© 2008 Progress Software Corporation32
Agenda
High Availability• Problem Avoidability
• Visibility
• Scalability
Maintainability
The database “ilities”
© 2008 Progress Software Corporation33
Roll forward protection
myDb
ai ai ai ai ai ai ai ai
myDb.bak
© 2008 Progress Software Corporation34
Roll forward protection
myDb
ai ai ai ai ai ai ai ai
X** The database was last changed Mon Apr 1 15:37:38 2008.** The after-image file expected Mon Apr 1 15:33:45 2008.** Those dates don't match, so you have the wrong copy of one of them.roll forward open /usr1/x.a4 error: -1.
Roll forward
© 2008 Progress Software Corporation35
Roll forward protection
myDb
ai ai ai ai ai ai ai ai
XIn the .lg file:
rfutil -C roll forward session end. Single-user session begin for richb on /dev/pts/101. Begin Physical Redo Phase at 256 .
© 2008 Progress Software Corporation36
Roll forward protection
rfutil <db> -C roll forward oplock Prevents “stray” database connections
• Prostrct add allowed Automatic disablement
• At roll forward completion Explicit disablement
rfutil <db> -C roll opunlock• Recovers db
• Stops the roll forward process
Non interruptible roll forward
© 2008 Progress Software Corporation37
Roll forward protection
myDb
ai ai ai ai ai ai ai ai
- Access to the database during roll forward process is not allowed because it will modify the database. - Write access to the database will not be allowed until the roll forward operations have completed.
rfutil <db> -C roll forward oplock –a myDb.a1
Connection attempts:
© 2008 Progress Software Corporation38
Index Rebuild
Index rebuild packing factor• Max % of space used
• Avoids costly index block splits
proutil <db> -C idxbuild –pfactor <60 – 100>
Examine utilization % in idxanalys
More control
© 2008 Progress Software Corporation39
Index Fix Interface
Idxfix uses idxbuild interface• Select indexes to fix by
– Table, schema, area or activation state
Specific choices
Select one of the following:
All (a/A) - Fix all the indexes
Some (s/S) - Fix only some of the indexes
By Area (r/R) - Fix indexes in selected areas
By Schema (c/C) - Fix indexes by schema owners
By Table (t/T) - Fix indexes in selected tables
By Activation (v/V) - Fix selected active or inactive indexes
© 2008 Progress Software Corporation41
SQL Stored Procedures
64 bit JVM availability• Java™ 1.5 certification
• Can use same drivers
• Additional schema– _SysProcBin, _SysProcText
• 32 bit databases have schema already
• Databases created in 64 bit environmentproutil <db> -C enablestoredproc
64 bit stored procedure support.
© 2008 Progress Software Corporation42
Binary Dump Specified - Improved
proutil <db> -C dumpspecified <field-info>
<op1> <low-value>
AND <op2> <high-value> <dir>
Option values: GT, GE, LT, LE, EQ Dump specific ranges Improved parallelism
proutil db –C dumpspecified cust.custnum GE 5 and LE 100 /dumpdir
Binary dump specified with “between” range dumping
© 2008 Progress Software Corporation43
In Summary
We’ve made it even easier… to achieve high availability to see what’s going on to maintain
© 2008 Progress Software Corporation44
Relevant Exchange Sessions
OPS-1: How Healthy is Your Database Today?
OPS-18: Data Management Roadmap
OPS-19: What’s IPV6 and Why Should I Care?
OPS-28: A New Spin on Some Old Latches
© 2008 Progress Software Corporation45
Questions?
© 2008 Progress Software Corporation46
What’s here that is also in 10.1B03?
Database Consistency Checking• -memCheck, 0DbCheck, -AreaCheck, -IndexCheck, -
TableCheck Dbtool: Read or Validate Database Blocks Rfutil:
• aiverify• roll forward oplock
Promon:• Servers by Broker• Total Locks per User
Record lock removal on NO-LOCK reads
© 2008 Progress Software Corporation47
Thank You
© 2008 Progress Software Corporation48