dimensional modeling cs 543 – data warehousing. cs 543 - data warehousing (sp 2007-2008) - asim...

Dimensional Modeling

CS 543 – Data Warehousing

CS 543 - Data Warehousing (Sp 2007-2008) - Asim Karim @ LUMS 2

From Requirements to Data Models


Logical Data Model

Logical data design includes identification of all data elements and the structures in which they are connected Data elements Data structures

Requirement gathering, and more specifically, information packages lead to the logical data design


Dimensional Modeling

A logical data design technique to structure the business dimensions and the metrics that are analyzed along these dimensions

Dimensional modeling Is intuitive for business Has proven to be efficient for queries and analyses

Information packages are the foundation of dimensional modeling


Fact Table


Dimension Tables


Desired Characteristics in the Model

The model should provide the best data access The model should be query-centric The model must be optimized for queries and analyses The model must show that the dimension tables

interact with the fact table It must be structured in such a way that every

dimension can interact equally with the fact table The model should allow drill down and roll up along

dimension hierarchies


The Star Schema


E-R Vs. Dimensional Modeling (1)

Entity-relationship modeling Removes data redundancy Ensures data consistency Expresses microscopic relationships

Dimensional modeling Captures critical measures Views along dimensions Intuitive for business users


E-R Vs. Dimensional Modeling (2)

DM rules more restrictive than for E-R modeling.

DM is a simpler logical model. E-R representative power is greater due to

variety of constructs supported. DM looks like normalized E-R conceptual,

except:All relationships mandatory M-1.Single path between any two relations.


Another Example: Retail Dimensions

O pera tions P lann ingM erchand is ingA d /F isca l F isca l C a lendarA dvertis ing

Sales

S to re

D is tric t

R eg ion

M arke t

Z one

D ay

F isca l Y ea r

F isca l P e riod

F isca l Q uarte r

C a lenda r M on th

C a lenda r Y ea r

C a lenda r Q uarte r

C a lenda r W eek

U P C

S K U

S ubca tego ry

C a tegory

D epartm en t

A d W eek

A d M on th

A d Y ear

a ligned


Star Schema


Querying Against a Star Schema


Dimension Tables Characteristics

Dimension table key Large number of attributes (wide) Textual attributes Attributes not directly related Flattened out, not normalized Ability to drill down/roll up Multiple hierarchies Less number of records


Fact Table Characteristics

Concatenated fact table key Grain or level of data identified Fully additive measures Semi-additive measures Large number of records Only a few attributes Sparsity of data Degenerate dimensions


“Factless” Fact Table

If the metric or unit of analysis is occurrence or non-occurrence of an event, then the fact table will contain either 1 or nulls


Data Granularity (1)

Actual events are tied to actual transactions (e.g., sales). This corresponds to the lowest grain or highest detail

Accumulated events are the effect of accumulated transactions (e.g., inventory on hand). This corresponds to a higher grain and lesser detail

Allowable events represent the “ability” to perform a transaction (e.g., carried products, a.k.a. plan-o-gram).

Actual events are typically more sparse than allowable events (e.g., a store carries more products than it sells each day).


Data Granularity (2)

Low grain designs are easy to change (“graceful” change)

Low grain designs result in larger storage and maintenance costs


Keys (1)

Be careful in picking and using operational system keys as keys for the dimension tables Avoid built-in meanings in the primary keys of the

dimension tables Do not use operational system keys as primary keys of

dimension tables Use surrogate keys (system generated keys)

Keep a mapping between the surrogate and primary keys Include the operational system primary key as an attribute in

the dimension table


Keys (2)

Primary key options for fact tables A single compound primary key whose length is the

total length of the keys of the dimension tables Foreign keys need to be stored as additional attributes in the

fact table Increases size of fact table

A concatenated primary key that is the concatenation of all the primary keys of the dimension tables No need to store the foreign keys separately

A generated primary key independent of the keys of the dimension tables Foreign keys need to be stored as additional attributes


Advantages of the Star Schema

Easy for users to understand Optimizes navigation More suitable for query processing Star-join and star-index


Updates

Updates to the fact table Addition of rows Changes in row (adjustments in values) Rarely, addition of attributes (new fact or metric)

Updates to dimension tables Slow addition of rows Slow addition of attributes New dimensions


Updates to the Dimension Tables

Most dimensions are generally constant over time Many dimensions, if not constant, change slowly over

time The key of the source record does not change The description and other attributes change slowly over

time In the source OLTP systems, the new values overwrite

the old values Overwriting is not always the best option for

dimension table attributes The way updates are made depends on the type of

change


Type 1 Changes: Correction of Errors (2)

Overwrite the attribute value in the dimension table row with the new value

The old value of the attribute is not preserved No other change are made in the dimension table row The key of this row or any other key value are not

affected This type is easiest to implement


Type 1 Changes (2)


Type 2 Changes: Preservation of History (1)

Properties They usually relate to true changes in source systems There is a need to preserve history in the data warehouse This type of change partitions the history in the data warehouse Every change for the same attribute must be preserved

Approach Add a new dimension table row with the new value of the changed

attribute An effective data field may be added into the dimension table There are no changes to the original row of the dimension table The new row is inserted with a new surrogate key The key of the original row is not affected


Type 2 Changes: Preservation of History (2)


Type 3 Changes: Tentative Soft Revisions (1)

Properties They usually apply to “soft” or tentative changes in the source systems There is a need to keep track of history with old and new values of the

changed attribute They are used to compare performances across the transition They provide the ability to track forward and backward

Approach Add an “old” field in the dimension table for the affected attribute Push down the existing value of the attribute from the “current” field to

the “old” field Keep the new value of the attribute in the “current” field Also, you may add a “current” effective date field for the attribute The key of the row is not affected No new dimension row is needed


Type 3 Changes: Tentative Soft Revisions (2)


Large Dimensions (1)

Large dimensions? Large number of rows (deep) Large number of attributes (wide)

Dimensions can become large because of frequent changes (what type?) and need to have many attributes for analysis

Consequence Slow and inefficient

Solution Proper logical and physical design Indexes Optimized algorithms


Large Dimensions (2)


Multiple Hierarchies


Junk Dimensions

Dimensions for a DW are typically taken from operational source systems

However, source systems contain many additional attributes (such as flags, text, descriptions, etc) that may not be useful in a DW

What are the options Discard all such fields in the source systems Include them in the fact table Include all of them as dimensions Select some and add them to a single “junk” dimension table


The Snowflake Schema

Snowflaking is a method of normalizing the tables in a star schema

dimensional modeling cs 543 – data warehousing. cs 543 - data warehousing (sp 2007-2008) - asim...

Documents

data warehousing slide

data warehousing sp

data redundancy

level of data

data models

asim karim

logical data design

dimensional modeling