bringing internals to the surface internals concepts for highly effective engines mark scranton...

Bringing Internals to Bringing Internals to the Surface the Surface

Internals Concepts for Highly Effective

Engines

Mark ScrantonPrincipal Consultant/Trainer

Informix Denverwww.markscranton.com

2

useruser .conference.conferenceInformixInformix

Who Am I?Who Am I?

• Principal Consultant/Trainer, Informix Denver

• I do engines - XPS and IDS

• 5+ yrs with Informix

• website - www.markscranton.com• tips/tricks• sql/scripts• documents, presentations

• Principal Consultant/Trainer, Informix Denver

• I do engines - XPS and IDS

• 5+ yrs with Informix

• website - www.markscranton.com• tips/tricks• sql/scripts• documents, presentations

3


Presentation OverviewPresentation Overview

• This presentation will identify “internals” concepts that appear to be trivia

• But they could be useful daily for maintenance and support of IDS

• A strong working knowledge of IDS is assumed

• More info available in the “IDS Internal Architecture” class

• This presentation will identify “internals” concepts that appear to be trivia

• But they could be useful daily for maintenance and support of IDS

• A strong working knowledge of IDS is assumed

• More info available in the “IDS Internal Architecture” class

4


TopicsTopics

• Physical pages vs. logical pages? • Discussion about the value of knowing

the difference

• The dog has swallowed it’s tail • ...or the tale of physical log overflow

• PARTNUMs - and the story of getting to data• We’ve all seen them…what does it really mean?

• Physical pages vs. logical pages? • Discussion about the value of knowing

the difference

• The dog has swallowed it’s tail • ...or the tale of physical log overflow

• PARTNUMs - and the story of getting to data• We’ve all seen them…what does it really mean?

5


TopicsTopics

• The maximum number of extents for a table? • Is there REALLY a max?

• Where did my row go?• The long tale of the forward pointer

• The last topic

• The maximum number of extents for a table? • Is there REALLY a max?

• Where did my row go?• The long tale of the forward pointer

• The last topic

6


Topic 1: Logical Pages vs. Physical PagesTopic 1: Logical Pages vs. Physical Pages

• Understanding the difference can help in:• Interpretation of msg log assertion failures• Interpretation of the extent list of a partition page• Lock level identification with respect to rowid(s)• Oncheck usage• A bunch of other stuff

• Understanding the difference can help in:• Interpretation of msg log assertion failures• Interpretation of the extent list of a partition page• Lock level identification with respect to rowid(s)• Oncheck usage• A bunch of other stuff

7


Physical PagesPhysical Pages

• Always with respect to a chunk

• Numbering starts with 0

• Format:

0xCCCPPPPP

ccc - chunk numberppppp - page offset into the chunk

• Always with respect to a chunk


• Format:

0xCCCPPPPP

ccc - chunk numberppppp - page offset into the chunk

8


Physical PagesPhysical Pages

30023c 30023d

300243300242300241300240

30023f30023e

300247300246300245300244

Chunk 3

0xCCCPPPPPc - chunk #p - page offset

9


Logical PagesLogical Pages

• Always with respect to a tablespace


• Format simply 0 through x

• Always with respect to a tablespace


• Format simply 0 through x

Logical PagesLogical Pages

30023c 30023d

300243300242300241300240

30023f30023e

300247300246300245300244

extent 1 for sparky

0

54

21

3

logicalpage num

create tablesparky...

physicalpage num

11


Usage Examples - Extent ListUsage Examples - Extent List

extent list on a partition page (oncheck -pt skippy:sparky)

chunk 0x4 page 0x33

12


Usage Examples - Bad PageUsage Examples - Bad Page

13


Usage Examples - Lock LevelUsage Examples - Lock Level

onstat -k

rowid format: 0xLLLLLLSSL - logical page in the tableS - slot/row # on the page

oncheck -pp 0x100016 2

14


Topic 2: Physical Log OverflowTopic 2: Physical Log Overflow

Physical Log File

1st before images of modified cache pages

75% full: checkpoint request is raised

what happens now?

back to the beginning, and keep writing!

15


Physical Log OverflowPhysical Log Overflow

Physical Log File

75% full: checkpoint request is raised

• Two potential scenarios:• the chkpt completes, and we give the physical log a

new logical beginning. In that case, we don’t care about the overwritten pages

• if the engine falls over, you’ll most likely get stuck in fast recovery when trying to come back up

• Two potential scenarios:• the chkpt completes, and we give the physical log a

new logical beginning. In that case, we don’t care about the overwritten pages

• if the engine falls over, you’ll most likely get stuck in fast recovery when trying to come back up

16


Topic 3: PartnumsTopic 3: Partnums

• Three areas focused on here:• What is a partnum? • How do we use it to get to stuff• What is a tablespace tablespace?

• Three areas focused on here:• What is a partnum? • How do we use it to get to stuff• What is a tablespace tablespace?

17


DefinitionsDefinitions

• Tablespace tablespace• Keeps track of tablespaces, or partitions,

in a dbspace• Each dbspace has one• Typical allocation is 50 pages for each extent

• Partition page• A page in a tablespace tablespace• Typically one page per tablespace• It “describes” the tablespace

• Tablespace tablespace• Keeps track of tablespaces, or partitions,

in a dbspace• Each dbspace has one• Typical allocation is 50 pages for each extent

• Partition page• A page in a tablespace tablespace• Typically one page per tablespace• It “describes” the tablespace

18


partnumpartnum

D D D L L L L L

dbspace number(starting at 1)

Logical page numwithin the tablespace tablespace (starting at 0)

1 1/2 bytes(3 nibbles, or 3 hex digits)

2 1/2 bytes(5 nibbles, or5 hex digits)

4 bytes

0x

19


dbspace4

rootdbs

systablesname partnumdatabase

sparky

0020000311-11-1999informixsparky

database tablespace

partnumflagscreatedownername

0020003cmudtable mud

extentsfor mud

tablespace tablespace page 3c

CREATE DATABASE sparky IN dbspace4;CREATE TABLE mud;

DATABASE sparky;SELECT * FROM mud;

20


Topic 4: Max Number of ExtentsTopic 4: Max Number of Extents

• What’s the maximum number of extents for a tablespace?

• Consider the previous topic, and the use of the partition page for a tablespace

• What’s the maximum number of extents for a tablespace?

• Consider the previous topic, and the use of the partition page for a tablespace

21


The Partition PageThe Partition Page

22


The Partition PageThe Partition Page

oncheck -pt skippy:sparky

23


Oh by the Way...Oh by the Way...

• Do you really want to know the max?

• Fragmentation changes the max for a logical table

• 4K vs. 2K pages

• Do you really want to know the max?

• Fragmentation changes the max for a logical table

• 4K vs. 2K pages

24


Topic 5: Forward PointersTopic 5: Forward Pointers

• Forward pointer - a 4-byte number that “points” to the remainder pieces of a row that spans pages

• Used with rows longer than a page

• Format is same as a rowid, although it’s NOT the rowid for the row

• Used with varchars or really long rows (rows longer than a page typically)

• We’ll focus on varchars ...

• Forward pointer - a 4-byte number that “points” to the remainder pieces of a row that spans pages

• Used with rows longer than a page

• Format is same as a rowid, although it’s NOT the rowid for the row

• Used with varchars or really long rows (rows longer than a page typically)

• We’ll focus on varchars ...

25


Forward Pointers w/ varcharsForward Pointers w/ varchars

• Problem - you can easily end up with a “chain” of forward pointer/remainder portions of rows

• This can happen when you’ve updated a varchar column and increased it’s size

• The new row size won’t fit into it’s old slot, and so we:• Move the row - all or some of it - to another page• Leave back a 4-byte forward pointer

on the home page

• Problem - you can easily end up with a “chain” of forward pointer/remainder portions of rows

• This can happen when you’ve updated a varchar column and increased it’s size

• The new row size won’t fit into it’s old slot, and so we:• Move the row - all or some of it - to another page• Leave back a 4-byte forward pointer

on the home page

26


Forward Pointers w/ varchars Forward Pointers w/ varchars

• This can happen as many times as it needs

• Every move adds an additional 8 bytes to the row length• 4 bytes for the forward pointer in the home page• 4 bytes for the slot table entry on the

new/remainder page

• This can happen as many times as it needs

• Every move adds an additional 8 bytes to the row length• 4 bytes for the forward pointer in the home page• 4 bytes for the slot table entry on the

new/remainder page

27


Forward PointersForward Pointers

28


What Can I Do? What Can I Do?

• To rid yourself of the “chaining” effect:• Unload/reload the table

• If the row will fit onto a single page, then there is no need for the forward pointer

• If it’s grown to be larger than a page (actual row length), then you’ll have at least one. This can’t be avoided - the row is now longer than a page

• Alter index to cluster• Same caveats as above

• To rid yourself of the “chaining” effect:• Unload/reload the table

• If the row will fit onto a single page, then there is no need for the forward pointer

• If it’s grown to be larger than a page (actual row length), then you’ll have at least one. This can’t be avoided - the row is now longer than a page

• Alter index to cluster• Same caveats as above

29


Topic 7: How Do You Find Out More? Topic 7: How Do You Find Out More?

• Attend the “IDS Internal Architecture” class• 5 days - very intense use of oncheck, and topics

covered in this presentation• Taught in Denver, Chicago, Menlo Park most often

• Watch TechNotes for a full white paper on this topic

• Send me email - [email protected]

• Checkout the website - www.markscranton.com

• Attend the “IDS Internal Architecture” class• 5 days - very intense use of oncheck, and topics

covered in this presentation• Taught in Denver, Chicago, Menlo Park most often

• Watch TechNotes for a full white paper on this topic

• Send me email - [email protected]

• Checkout the website - www.markscranton.com

30


Thanks...Thanks...

• For listening. Hope you learned something

• See you in class

• For listening. Hope you learned something

• See you in class

bringing internals to the surface internals concepts for highly effective engines mark scranton...

Documents