Dimensional Modeling – Part 2

CS 543 – Data Warehousing

CS 543 - Data Warehousing (Sp 2007-2008) - Asim Karim @ LUMS 2

The Snowflake Schema

Snowflaking is a method of normalizing the dimension tables in a star schema

Normalization increases the efficiency of certain queries, and reduces space requirements

CS 543 - Data Warehousing (Sp 2007-2008) - Asim Karim @ LUMS 3

Star Schema

CS 543 - Data Warehousing (Sp 2007-2008) - Asim Karim @ LUMS 4

Querying

Suppose the product table has 500,000 rows (different products). These products fall under 500 product brands, and these brands fall under 10 product categories

Query: give me the total quantity of a specific product category sold in Jan 2004? All 500,000 rows in the product dimension table would have

to be searched to find the products belonging to the specified product category

CS 543 - Data Warehousing (Sp 2007-2008) - Asim Karim @ LUMS 5

A Snowflake Schema

CS 543 - Data Warehousing (Sp 2007-2008) - Asim Karim @ LUMS 6

Normalization

Partially or fully normalize only a few dimension tables, leaving the others intact

Partially normalize every dimension table Fully normalize every dimension table

CS 543 - Data Warehousing (Sp 2007-2008) - Asim Karim @ LUMS 7

Snowflaking?

Advantages Small savings in storage space Normalized structures are easier to update and maintain

Disadvantages Schema less intuitive and end-users are put off by the

complexity Ability to browse through the contents difficult Degraded query performance because of additional

joins

CS 543 - Data Warehousing (Sp 2007-2008) - Asim Karim @ LUMS 8

Sub-dimensions

CS 543 - Data Warehousing (Sp 2007-2008) - Asim Karim @ LUMS 9

Some Query Examples (1)

CS 543 - Data Warehousing (Sp 2007-2008) - Asim Karim @ LUMS 10

Some Query Examples (2)

Query: Total sales for customer number 12345678 during the first week of December 2003 for product Widget-1

Find and sum the sales quantity and sales dollars for all fact table rows where the customer key relates to customer number 12345678, the product key relates to product Widget-1, and the time key relates to the seven days in the first week of December 2003. Assuming a customer can make a single purchase on a given

day, only seven rows of the fact table will be summed

CS 543 - Data Warehousing (Sp 2007-2008) - Asim Karim @ LUMS 11

Some Query Examples (3)

Query: total sales for all customers in the south-central territory for the first two quarters of 2003 for product category Bigtools

All fact table rows where the customer key relates to all customers in the south-central territory, the product key relates to all products in the product category Bigtools and the time key relates to about 180 days in the first two quarters of 2003. In this query, clearly a large number of fact table rows

participate the summation

How can we reduce the execution time?

CS 543 - Data Warehousing (Sp 2007-2008) - Asim Karim @ LUMS 12

Fact Table Size (1)

CS 543 - Data Warehousing (Sp 2007-2008) - Asim Karim @ LUMS 13

Fact Table Size (2)

Credit card transaction tracking

Time dimension: 5 years = 60 months Number of credit card accounts: 150 million Av. number of monthly transaction/account: 20 Max. number of base fact table records: 180 billion

CS 543 - Data Warehousing (Sp 2007-2008) - Asim Karim @ LUMS 14

Aggregating Fact Table

Typically, queries require detailed data on some dimensions, while only summary data is needed for the other dimensions

Example: assume one sale per product per store per week. Estimate the number of fact table rows required: Query involves 1 product, 1 store, 1 week Query involves 1 product, all stores, 1 week Query involves 1 brand, 1 store, 1 week Query involves 1 brand, all stores, 1 year Suppose now you have an aggregate fact table where each

row summarizes the totals for a brand, a store, and a week. Now estimate the number of fact table rows required.

CS 543 - Data Warehousing (Sp 2007-2008) - Asim Karim @ LUMS 15

Multi-Way Aggregate Fact Tables (1)

Utilize hierarchies in dimensions to create appropriate aggregate fact tables

Single-way aggregate fact table aggregates along one dimension only; multi-way have more than one dimension aggregated

CS 543 - Data Warehousing (Sp 2007-2008) - Asim Karim @ LUMS 16

Multi-Way Aggregate Fact Tables (2)

CS 543 - Data Warehousing (Sp 2007-2008) - Asim Karim @ LUMS 17

Multi-Way Aggregate Fact Tables (3)

CS 543 - Data Warehousing (Sp 2007-2008) - Asim Karim @ LUMS 18

Goals for Aggregation

Primary goal: improve overall DW performance Do not get bogged down with too many aggregates.

Remember you have to create addition derived dimensions to support the aggregates

Try to cater to a wide range of user groups Go for aggregates that do not unduly increase the

overall usage of storage Keep the aggregates hidden from the end-users

CS 543 - Data Warehousing (Sp 2007-2008) - Asim Karim @ LUMS 19

Families of Stars

CS 543 - Data Warehousing (Sp 2007-2008) - Asim Karim @ LUMS 20

Snapshot and Transaction Tables

CS 543 - Data Warehousing (Sp 2007-2008) - Asim Karim @ LUMS 21

Conformed Dimensions

Since multiple fact tables share dimension tables, it is essential that dimensions are conformed, i.e., they have the same meaning

Conformed dimensions are essential for Building up an enterprise warehouse from data marts Running queries across data marts Consistent semantics of queries and their results

Using conformed dimensions is a important responsibility of the DW project team

CS 543 - Data Warehousing (Sp 2007-2008) - Asim Karim @ LUMS 22

Standardizing Facts

Since fact tables can be shared, they need to be standardized. Ensure same definition and terminology across data marts Resolve homonyms and synonyms Guarantee that the same algorithms are used for any derived

units in each fact table Make sure each fact uses the right unit of measurement