a_blueprint_of_sanssoucidb.pdf

8/19/2019 a_blueprint_of_sanssouciDB.pdf

1/6

Prof. Hasso Plattner

A Course in

In-Memory Data ManagementThe Inner Mechanicsof In-Memory Databases

September 4, 2015

This learning material is part of the reading material for Prof.Plattner’s online lecture "In-Memory Data Management" taking place atwww.openHPI.de. If you have any questions or remarks regarding theonline lecture or the reading material, please give us a note at [email protected]. We are glad to further improve the material.

Research Group "Enterprise Platform and Integration Concepts",http: // epic.hpi.uni-potsdam.de

http://www.openhpi.de/http://epic.hpi.uni-potsdam.de/http://epic.hpi.uni-potsdam.de/http://epic.hpi.uni-potsdam.de/http://epic.hpi.uni-potsdam.de/http://www.openhpi.de/


2/6


3/6

Chapter 5

A Blueprint of SanssouciDB

SanssouciDB is a prototypical database system for unified analytical andtransactional processing. The concepts of SanssouciDB build on prototypesdeveloped at the HPI and an existing SAP database system. SanssouciDB is

an SQL database and it contains similar components as other databases suchas a query builder, a plan executer, meta data, a transaction manager, etc.

5.1 Data Storage in Main Memory

In contrast to traditional database management systems, the primary persis-tence of SanssouciDB is main memory. Yet logging and recovery still requiredisks as non-volatile data storage to ensure data consistency in case of fail-ures. All operators, e.g., find, join, or aggregation can anticipate that dataresides in main memory. Thus, operators are implemented diff erently mov-ing the focus from optimizing for disk access towards optimizing for mainmemory access and CPU utilization (see Chapter 4).

This apparently subtle diff erence of moving the primary persistence hasa vast impact on performance even when disk-based databases are com-pletely memory resistent. Ailamaki et al. invested such fully cached disk-

based databases and found that a large portion of query execution is spentfor memory and resource stalls [ADHW99]. Those stalls are mainly caused

by in-page data placements that do not utilize the CPU caches properly.In many cases, the actual computation accounts for less than 40% of theexecution time. Besides, Harizopoulos et al. found that the buff er manage-ment of disk-based databases alone contributes 31% to the overall instructioncount [HAMS08]

Consequently, the reason for the performance advantages of in-memory

over disk-based databases derives from optimized data structures and algo-rithms avoiding memory and resource stalls together with the removal of additional indirections.

33

http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-


4/6

34 5 A Blueprint of SanssouciDB

5.2 Column-Orientation

Another concept used in SanssouciDB was invented more than two decadesago, that is, storing data column-wise [CK85] instead of row-wise. In column-

orientation, complete columns are stored in adjacent blocks. This can be con-trasted with row-oriented storage where complete tuples (rows) are stored inadjacent blocks. Column-oriented storage, in contrast to row-oriented stor-age, is well suited for reading consecutive entries from a single column. Thiscan be useful for aggregation and column scans. More details on column-orientation and its diff erences to row-orientation can be found in Chapter 8.To minimize the amount of data that needs to be transferred between stor-age and processor, SanssouciDB uses several diff erent data compressiontechniques, which will be discussed in Chapter 7.

5.3 Implications of Column-Orientation

Column-oriented storage has become widespread in database systemsspecifically developed for OLAP, as the advantage of column-oriented stor-age is clear in case of quasi-sequential scanning of single attributes and setprocessing thereof. If not all fields of a table are queried, column-orientationcan be exploited as well in transactional processing (avoiding "SELECT *").An analysis of enterprise applications showed that there is actually no ap-plication that uses all fields of a given tuple. For example, in dunning only17 attributes are necessary out of a table that contains 300 attributes. If onlythe 17 needed attributes are queried instead of the full tuple representationof all 300 attributes, an instant advantage of factor eight to 20 for data to bescanned can be achieved.

As disk is not the bottleneck any longer, but access to main memory has to be considered, an important aspect is to work on a minimal set of data. So far,application programmers were fond of "SELECT *" statements. The diff er-ence in runtime between selecting specific fields or all fields in row-orientedstorage is often insignificant and in case changes to an application needmore fields, the data was already there (which besides is a weak argumentfor using SELECT * and retrieving unnecessary data). However, in case of column-orientation, the penalty for "SELECT *" statements grows with tablewidth. Especially if tables are growing in width during productive usage,actual runtimes of applications cannot be anticipated during programming.

With the column-store approach, the number of indices can be signifi-cantly reduced. In a column store, every attribute can be used as an index.Because all data is available in memory and the data of a column is stored

consecutively, the scanning speed is high enough that a full sequential scanof an attribute is sufficient in most cases. If this is not fast enough, dedicatedindices can still be used in addition for further speedup.



5/6

5.4 Architecture Overview 35

Storing data in columns instead of rows is challenging for workloadswith many data modifying operations. Therefore, the concept of a diff eren-tial buff er was introduced, where new entries are written to a diff erential

buff er first. In contrast to the main store, the diff erential buff er is optimized

for inserts. At a later point in time and depending on thresholds, e.g. thefrequency of changes and new entries, the data in the diff erential buff er ismerged into the main store. More details about the diff erential buff er andthe merge process will be provided later in Chapter 25 and Chapter 27.

5.4 Architecture Overview

The architecture shown in Figure 5.1 grants an overview of the componentsof SanssouciDB.

SanssouciDB is split in three diff erent logical layers fulfilling specific tasksinside the database system. The “Management Layer” handles the commu-

nication to applications, creates query execution plans, stores meta data andcontains the logic fordatabase transactions. Insidethe main memory of a spe-cific machine the main working set of SanssouciDB is located. That workingset is accessed during query execution and is stored either in row, column orhybrid-oriented data layout, depending on the specific type of queries sentto the database tables. The non-volatile memory in the durable storage areais used for logging and recovery purposes, as well as for data aging and timetravel. All those concepts will be described in the subsequent sections.

. .

: .

Read-only ReplicasRead-only Replicas

Hot Store (Master)

M e r g e

Delta

Cold Store - 1

Main MemoryStorage

DurableStorageLog

Query Execution Metadata Sessions Transactions

ManagementLayer

Financials Logistics Manu-

facturing …

History

…

Main

AggregateCache

Attribute Vectors

Dictionaries

Index

Dictionaries

Index

Attribute Vectors

Cold Store - 2

Main

Aggregate Cache

Attribute Vectors

Dictionaries

Index

OLTP & OLAP Applications

Main

Aggregate Cache

Attribute Vectors

Dictionaries

Index

CheckpointCheckpoints

Stored Procedures

SQL Interface

Fig. 5.1: Schematic Architecture of SanssouciDB



6/6

36 REFERENCES

5.5 References

[ADHW99] Anastassia Ailamaki, David J. DeWitt, Mark D. Hill, andDavid A. Wood. Dbmss on a modern processor: Where does

time go? In Malcolm P. Atkinson, Maria E. Orlowska, PatrickValduriez, Stanley B. Zdonik, and Michael L. Brodie, editors,VLDB, pages 266–277, San Francisco, CA, USA, 1999. MorganKaufmann.

[CK85] George P. Copeland and Setrag N. Khoshafian. A Decomposi-tion Storage Model. SIGMOD Rec., 14(4):268–279, May 1985.

[HAMS08] Stavros Harizopoulos, Daniel J. Abadi, Samuel Madden, andMichael Stonebraker. Oltp through the looking glass, and whatwe found there. In Jason Tsong-Li Wang, editor, SIGMOD Con-

ference, pages 981–992. ACM, 2008.

a_blueprint_of_sanssoucidb.pdf

Documents