Concrete Architectureof PostgreSQL

S-Queue-LKhurrum A Mujeeb, Adam Abu Hijleh, Adam Ali

Stephen McDonald, Wisam Zaghal

CISC 322 - Fall 2010

Overview– Derivation Process – Conceptual Architecture Revisited– High Level Conceptual Dependencies– High Level Observed Dependencies (Concrete Architecture)

• Expose high level unexpected inter-dependencies

– Utilities• Intradependencies• Interdependencies

– Final Arcihtecture Style & Conceptual Modification– Use Case Revisted– Concurrency & Team Issues– Limitations– Lessons Learned– Conclusion

Derivation Process– Map components to our proposed conceptual architecture

• With aid of OpenGrok online source code browser, Software Landscape Editor, online PostgreSQL Manual, and source code decomposition

– Analysis of dependencies for the now-grouped components, making note of missing & unexpected dependencies

– Investigate gaps between conceptual dependencies & extracted concrete dependencies (Reflextion Analysis)

Propose

Conceptual subsystems

Dependencies between

SubSystems

Mapping source entities to

subsystems

Extracted source dependencies

Conceptual Architecture

Concrete Architecture

Compare

GapsInvestigate

Reflexion Analysis - Conceptual

Figure 1

Conceptual Architecture Revisited - Layered

Legend

Expected

Dependency

Figure 2

Expected Subsystem Interdependencies

ExpectedSubsystem Interdependency

Legend

Figure 3

Propose

Conceptual subsystems

Dependencies between

SubSystems

Mapping source entities to

subsystems

Extracted source dependencies

Conceptual Architecture

Concrete Architecture

Compare

GapsInvestigate

Reflexion Analysis - Concrete

Figure 4

Unexpected Subsystem Interdependencies

ExpectedSubsystem Interdependency

Legend

UnexpectedSubsystem Interdependency

Figure 5

Legend

Expected

Dependency

Unexpected

Dependency

Figure 6

Unexpected Subsystem Interdependencies

pl

test

tutorial

include

tools

snowball

tsearch

regex

port

foreign

bin

Miscellaneous

Query Processor

Process Manager

Client Communication

Storage Manager

Catalog

GeneralUtils

Nodes/List

Access

Utility Inter-Subsystem Dependencies

Unexpected Dependencies

All External Subsystems Depend on

Expected Dependency

Query Processor dependency

Process Manager

Client Comm. Dependency

Storage Manager

Utility Component

Subsystems

Figure 7

Catalog

GeneralUtils

Nodes/List

Access

Utility Intra-Subsystem Dependencies

Unexpected Dependencies Utility component

Figure 8

Backend Utilities

Time zone Library Numeric.c

Unexpected Dependencies Utility component

General Utilities Intra-Subsystem Dependency

Figure 9

Propose

Conceptual subsystems

Dependencies between

SubSystems

Mapping source entities to

subsystems

Extracted source dependencies

Conceptual Architecture

Concrete Architecture

Compare

GapsInvestigate

Reflexion Analysis – Investigate Gaps

Figure 10

Architecture Style Revisited

Initially:Layered– Each layer only talks to the layer above or below

Maybe: Minimally layered ?- too much coupling

Now:Object Oriented – Function calls within and across various subsystems

High Level Conceptual Remodeling – Object Oriented

Figure 11

NewExpected Dependencies Subsystem

ClientLibrary Interfac

eServer

Parser Stage

Rewriter

Planner/ Optimizer

Executor

StorageManage

r

Request toLog in Request to

Log in

Server and communi-cation channel created

Logged in

QuerySent

QuerySent Query

Sent QuerySent

ExecutedQueryReturned

ExecutedQueryReturned

ExecutedQueryReturned

CatalogGeneralUtilities

Nodes

ServerRequested

Grammar rules and actions looked up

Grammar rules and actions retrieved

Copy node tree function called

Node tree returned

Tree manipulation function called

Manipulated tree returned

Tree comparison function called

Tree manipulation function called

Manipulated tree returned

ParsedTree

ModifiedTree

MostEfficientTree sent Access & Modify

Tree

Semantics lookup

Semantics retrieved

Comparison results returned

Format type function calledSQL format language returned

DataSent back

Current execution state of the queryExecuted Query Returned

Legend:

Components

Data Flow

Function Call

Duration ofrunning component

User

Figure 12

Concurrency &Team Issues

Team Issues- CommitFests

- Review and commit patches if up to par- If not committed, feedback given and changes

made

Concurrency– MVCC

• snapmgr.c (Utils - time)– Snapshot of data

• proc.c (Storage – Lock Manager)– Frees lock associated with current transaction

Limitations– Software Landscape Editor, although powerful, was primitive –

hard to use

– Determining how to map source files in accordance with our conceptual architecture

– Lack of documentation on important code fragments

– Was difficult to conslidate utilities into one shared component, as utilities were scattered throughout the various levels of the system

Lessons Learned

– Best way to understand High Level dependencies is to understand their origins on the lower levels

– Importance of clear and efficient commenting

– Group effort needed at every stage– Divide and Conquer proved to be much more difficult

given the nature of the contain files and Lsedit– Working in pairs of 2 at a time per task proved to be

most effective

Conclusions

– All components were much more intertwined than originally suspected

– Importance of reflexion analysis

– We now believe the architecture to be Object-Orientated