Organization Normalization: Another Example OLTP, OLAP, SAP, and Data Warehouse SQL: Subtleties for COUNT and JOINs

InformationslogistikUnit 10: OLTP, OLAP, SAP, Data Warehouse, and

Object-relational Databases

Ronald Ortner

19. V. 2015

Ronald Ortner

Organization Normalization: Another Example OLTP, OLAP, SAP, and Data Warehouse SQL: Subtleties for COUNT and JOINs

Outline

1 Organization

2 Normalization: Another Example

3 OLTP, OLAP, SAP, and Data WarehouseOLTP and OLAPSAPData Warehouse & OLAP

4 SQL: Subtleties for COUNT and JOINs

Ronald Ortner

Organization Normalization: Another Example OLTP, OLAP, SAP, and Data Warehouse SQL: Subtleties for COUNT and JOINs

Organization

Results for second intermediate test.Common errors:- see common errors sheet- when to use a correlated subqueryNo lectures next Tuesday, exercises due till June 2nd.Additional material for normalization:Chapter 2 ofAndreas Meier: “Relationale und postrelationale Datenbanken”,Springer (available online within MUL)

Ronald Ortner

Organization Normalization: Another Example OLTP, OLAP, SAP, and Data Warehouse SQL: Subtleties for COUNT and JOINs

Normalization: Another Example

Design a database for storing information of a ticket agency forpop/rock concerts. Store for each concert the band(s) playing at theconcert, date, country, town, and venue when/where the concert takesplace, the time when the concert starts, and the ticket price. (You mayassume that for a given concert all tickets are the same price.) Storealso for each customer buying tickets his/her name, address, andthenumber of tickets purchased for which concert(s).

Ronald Ortner

Organization Normalization: Another Example OLTP, OLAP, SAP, and Data Warehouse SQL: Subtleties for COUNT and JOINs

OLTP and OLAP

Outline

1 Organization

2 Normalization: Another Example

3 OLTP, OLAP, SAP, and Data WarehouseOLTP and OLAPSAPData Warehouse & OLAP

4 SQL: Subtleties for COUNT and JOINs

Ronald Ortner

Organization Normalization: Another Example OLTP, OLAP, SAP, and Data Warehouse SQL: Subtleties for COUNT and JOINs

OLTP and OLAP

OLTP vs. OLAP

OLTP: online transaction processingDatabase applications for ongoing workExamples: orders, bookings, etc.current data is important→ many updates and changes in database

OLAP: online analytical processingDatabase applications for analysis and decision supportExample: analysis of trendshistorical data is important→ lots of data, need information in aggregated form

Ronald Ortner

Organization Normalization: Another Example OLTP, OLAP, SAP, and Data Warehouse SQL: Subtleties for COUNT and JOINs

OLTP and OLAP

OLTP vs. OLAP

OLTP: online transaction processingDatabase applications for ongoing workExamples: orders, bookings, etc.current data is important→ many updates and changes in database

OLAP: online analytical processingDatabase applications for analysis and decision supportExample: analysis of trendshistorical data is important→ lots of data, need information in aggregated form

Ronald Ortner

Organization Normalization: Another Example OLTP, OLAP, SAP, and Data Warehouse SQL: Subtleties for COUNT and JOINs

SAP

Outline

1 Organization

2 Normalization: Another Example

3 OLTP, OLAP, SAP, and Data WarehouseOLTP and OLAPSAPData Warehouse & OLAP

4 SQL: Subtleties for COUNT and JOINs

Ronald Ortner

Organization Normalization: Another Example OLTP, OLAP, SAP, and Data Warehouse SQL: Subtleties for COUNT and JOINs

SAP

SAP

SAP: software system, mainly for OLTP

SAP has three levels:big relational database system in the backgroundapplications that work on the database systemgraphical user interface

Access to underlying database system:Some tables can be accessed also outside SAP (using SQL).Usually only read access is sensible.Some other tables can be accessed only via SAP.

Ronald Ortner

Organization Normalization: Another Example OLTP, OLAP, SAP, and Data Warehouse SQL: Subtleties for COUNT and JOINs

SAP

SAP

SAP: software system, mainly for OLTP

SAP has three levels:big relational database system in the backgroundapplications that work on the database systemgraphical user interface

Access to underlying database system:Some tables can be accessed also outside SAP (using SQL).Usually only read access is sensible.Some other tables can be accessed only via SAP.

Ronald Ortner

Organization Normalization: Another Example OLTP, OLAP, SAP, and Data Warehouse SQL: Subtleties for COUNT and JOINs

SAP

SAP

SAP: software system, mainly for OLTP

SAP has three levels:big relational database system in the backgroundapplications that work on the database systemgraphical user interface

Writing applications with ABAP/4 – access to databases withNative SQL (using special interface)Open SQL (direct access to databases)

Ronald Ortner

Organization Normalization: Another Example OLTP, OLAP, SAP, and Data Warehouse SQL: Subtleties for COUNT and JOINs

Data Warehouse & OLAP

Outline

1 Organization

2 Normalization: Another Example

3 OLTP, OLAP, SAP, and Data WarehouseOLTP and OLAPSAPData Warehouse & OLAP

4 SQL: Subtleties for COUNT and JOINs

Ronald Ortner

Organization Normalization: Another Example OLTP, OLAP, SAP, and Data Warehouse SQL: Subtleties for COUNT and JOINs

Data Warehouse & OLAP

OLTP vs. OLAP

OLTP: online transaction processingDatabase applications for ongoing workExamples: orders, bookings, etc.current data is important→ many updates and changes in database

OLAP: online analytical processingDatabase applications for analysis and decision supportExample: analysis of trendshistorical data is important→ lots of data, need information in aggregated form

→ no good idea to do OLTP and OLAP on the same database system

Ronald Ortner

Organization Normalization: Another Example OLTP, OLAP, SAP, and Data Warehouse SQL: Subtleties for COUNT and JOINs

Data Warehouse & OLAP

OLTP vs. OLAP

OLTP: online transaction processingDatabase applications for ongoing workExamples: orders, bookings, etc.current data is important→ many updates and changes in database

OLAP: online analytical processingDatabase applications for analysis and decision supportExample: analysis of trendshistorical data is important→ lots of data, need information in aggregated form

→ no good idea to do OLTP and OLAP on the same database system

Ronald Ortner

Organization Normalization: Another Example OLTP, OLAP, SAP, and Data Warehouse SQL: Subtleties for COUNT and JOINs

Data Warehouse & OLAP

Data Warehouse

Idea of Data Warehouse:Do OLTP on operational databasesStore information from operational databases regularly(but not online!) in data warehouse

Database Scheme for Data Warehouse:Star Scheme:

Central ‘fact’ tableother tables not normalized

Snowflake Scheme:Central ‘fact’ tableother tables normalized (→ more joins necessary)

Ronald Ortner

Organization Normalization: Another Example OLTP, OLAP, SAP, and Data Warehouse SQL: Subtleties for COUNT and JOINs

Data Warehouse & OLAP

Data Warehouse

Idea of Data Warehouse:Do OLTP on operational databasesStore information from operational databases regularly(but not online!) in data warehouse

Database Scheme for Data Warehouse:Star Scheme:

Central ‘fact’ tableother tables not normalized

Snowflake Scheme:Central ‘fact’ tableother tables normalized (→ more joins necessary)

Ronald Ortner

Organization Normalization: Another Example OLTP, OLAP, SAP, and Data Warehouse SQL: Subtleties for COUNT and JOINs

Data Warehouse & OLAP

Data Warehouse

Idea of Data Warehouse:Do OLTP on operational databasesStore information from operational databases regularly(but not online!) in data warehouse

Database Scheme for Data Warehouse:Star Scheme:

Central ‘fact’ tableother tables not normalized

Snowflake Scheme:Central ‘fact’ tableother tables normalized (→ more joins necessary)

Ronald Ortner

Organization Normalization: Another Example OLTP, OLAP, SAP, and Data Warehouse SQL: Subtleties for COUNT and JOINs

Data Warehouse & OLAP

Roll Up and Drill Down

Queries on Data Warehouse for analysis usually aggregate data(→ GROUP BY)

Drill down: more attributes in GROUP BY

Roll up: fewer attributes in GROUP BY

Data can be summarized in a cross table (data cube)

Ronald Ortner

Organization Normalization: Another Example OLTP, OLAP, SAP, and Data Warehouse SQL: Subtleties for COUNT and JOINs

Data Warehouse & OLAP

Relations for Aggregation & the Cube Operator

Creating the data cube:

expensive to execute all queries for creating cube

can store relation for data cube(using NULL values where aggregated)still elaborate and uncomfortable→ idea: new SQL operator CUBEUsage: GROUP BY CUBE( attr1, attr2, . . . )Other possibility:

storing maximally drilled-down tableaggregate this table(cheaper than doing each aggregation from scratch)

Ronald Ortner

Organization Normalization: Another Example OLTP, OLAP, SAP, and Data Warehouse SQL: Subtleties for COUNT and JOINs

Data Warehouse & OLAP

Relations for Aggregation & the Cube Operator

Creating the data cube:

expensive to execute all queries for creating cubecan store relation for data cube(using NULL values where aggregated)

still elaborate and uncomfortable→ idea: new SQL operator CUBEUsage: GROUP BY CUBE( attr1, attr2, . . . )Other possibility:

storing maximally drilled-down tableaggregate this table(cheaper than doing each aggregation from scratch)

Ronald Ortner

Organization Normalization: Another Example OLTP, OLAP, SAP, and Data Warehouse SQL: Subtleties for COUNT and JOINs

Data Warehouse & OLAP

Relations for Aggregation & the Cube Operator

Creating the data cube:

expensive to execute all queries for creating cubecan store relation for data cube(using NULL values where aggregated)still elaborate and uncomfortable

→ idea: new SQL operator CUBEUsage: GROUP BY CUBE( attr1, attr2, . . . )Other possibility:

storing maximally drilled-down tableaggregate this table(cheaper than doing each aggregation from scratch)

Ronald Ortner

Organization Normalization: Another Example OLTP, OLAP, SAP, and Data Warehouse SQL: Subtleties for COUNT and JOINs

Data Warehouse & OLAP

Relations for Aggregation & the Cube Operator

Creating the data cube:

expensive to execute all queries for creating cubecan store relation for data cube(using NULL values where aggregated)still elaborate and uncomfortable→ idea: new SQL operator CUBEUsage: GROUP BY CUBE( attr1, attr2, . . . )

Other possibility:storing maximally drilled-down tableaggregate this table(cheaper than doing each aggregation from scratch)

Ronald Ortner

Organization Normalization: Another Example OLTP, OLAP, SAP, and Data Warehouse SQL: Subtleties for COUNT and JOINs

Data Warehouse & OLAP

Relations for Aggregation & the Cube Operator

Creating the data cube:

expensive to execute all queries for creating cubecan store relation for data cube(using NULL values where aggregated)still elaborate and uncomfortable→ idea: new SQL operator CUBEUsage: GROUP BY CUBE( attr1, attr2, . . . )Other possibility:

storing maximally drilled-down tableaggregate this table(cheaper than doing each aggregation from scratch)

Ronald Ortner

Organization Normalization: Another Example OLTP, OLAP, SAP, and Data Warehouse SQL: Subtleties for COUNT and JOINs

Data Warehouse & OLAP

Row Store vs. Column Store

Usually, tables are stored row-wise.

When there are many columns, it may be better to storecolumn-wise:

Most queries consider only few columns.Column values can be better compressed.Use e.g. dictionary table.

Ronald Ortner

Organization Normalization: Another Example OLTP, OLAP, SAP, and Data Warehouse SQL: Subtleties for COUNT and JOINs

Data Warehouse & OLAP

Row Store vs. Column Store

Usually, tables are stored row-wise.When there are many columns, it may be better to storecolumn-wise:

Most queries consider only few columns.Column values can be better compressed.Use e.g. dictionary table.

Ronald Ortner

Organization Normalization: Another Example OLTP, OLAP, SAP, and Data Warehouse SQL: Subtleties for COUNT and JOINs

Data Warehouse & OLAP

Row Store vs. Column Store

Usually, tables are stored row-wise.When there are many columns, it may be better to storecolumn-wise:

Most queries consider only few columns.

Column values can be better compressed.Use e.g. dictionary table.

Ronald Ortner

Organization Normalization: Another Example OLTP, OLAP, SAP, and Data Warehouse SQL: Subtleties for COUNT and JOINs

Data Warehouse & OLAP

Row Store vs. Column Store

Usually, tables are stored row-wise.When there are many columns, it may be better to storecolumn-wise:

Most queries consider only few columns.Column values can be better compressed.

Use e.g. dictionary table.

Ronald Ortner

Organization Normalization: Another Example OLTP, OLAP, SAP, and Data Warehouse SQL: Subtleties for COUNT and JOINs

Data Warehouse & OLAP

Row Store vs. Column Store

Usually, tables are stored row-wise.When there are many columns, it may be better to storecolumn-wise:

Most queries consider only few columns.Column values can be better compressed.Use e.g. dictionary table.

Ronald Ortner

Organization Normalization: Another Example OLTP, OLAP, SAP, and Data Warehouse SQL: Subtleties for COUNT and JOINs

SQL-Lesson

Today:Extensions I:counting with 0Extensions II:when to put conditions in the ON / WHERE part

Ronald Ortner