professional issues in computing...nuces, islamabad campus data warehousing - fall 2012 12 nuces,...

Data Warehousing(The Need, Importance & the Big Picture)

Naveed Iqbal, Assistant Professor

NUCES, Islamabad Campus(Lecture Slides Week # 1)

NUCES, Islamabad Campus Data Warehousing - Fall 2012 2

Why this Course?

The World is changing / (in fact changed)

Either change or Be left behind.

Missing the opportunities or going in thewrong direction has prevented us fromgrowing.

What is the right direction?

Harnessing the data, in the knowledgedriven economy.

Doing what can’t be or difficult to automate.


The Need of the Time

Drowning in data AND/BUT starving for

information.

Knowledge is power BUT Intelligence is

absolute/super power.


The Need of the Time

Data

Information

Knowledge

Intelligence

POWER

($/£)

Evolution of Information Systems


Business Intelligence


Visualization



Date Warehousing – the big picture

Data Warehouse Server

(Tier 1)

Data

Warehouse

Operational

Data Bases

Semistructured

Sources Query/Reporting

Data Marts

MOLAP

ROLAP

Clients

(Tier 3)

Tools

Meta

Data

Data sources

Data

(Tier 0)

IT

Users

Business

Users

Business Users

Data Mining

Archived

data

Analysis

OLAP Servers

(Tier 2)

Extract

Transform

Load

(ETL)

www data


Approach of the Course

Develop an understanding of the underlying RDBMSconcepts.

Apply these concepts to VLDB / DSS environmentsand understand where and why they break down?

Expose the differences between RDBMS and DataWarehouse in the context of VLDB.

Provide the basics of DSS tools such as OLAP, DataMining and demonstrate their applications.

Demonstrate the application of DSS concepts andlimitations of the OLTP concepts through labexercises.


Summary of the Course

Introduction & Background

Extract-Transform-Load (ETL)

Normalization & De-Normalization

Dimensional Modeling

Online Analytical Processing (OLAP)

Data Quality Management (DQM)

Need for Speed (Parallelism, Join and Indexing Techniques)

DWH Implementation Steps

Complete Implementation Case Study

Lab and Tool Usage

…


Books

Reference Books Golfarelli & Rizzi, Data Warehouse Design – Modern

Principles and Methodoligies, McGRAW-Hill

W. H. Inmon, Building the Data Warehouse,

John Wiley & Sons Inc., NY

R. Kimball, The Data Warehouse Toolkit,

John Wiley & Sons Inc., NY

A. Abdullah, “Data Warehousing for Beginners: Concepts

& Issues”.

Paulraj Ponniah, Data Warehousing Fundamentals, John

Wiley & Sons Inc., NY

. . .


Course Execution Plan

Lecturing / Discussions

Lab Work + Tutorials

Assignments / Case Studies

Projects

Marks Breakup:

Mid-I: 12% Quizzes: 6%

Mid-II: 13% Assignments/Case Study: 9%

Final*: 40% Projects*: 20%

* Mandatory (Missing means F)


Code of Conduct

Regularity Attendance criteria as per university policy

Punctuality No entry after 5 minutes from class start time (N/A for habitual late

comers)

Discipline ABSOLUTLY NO COMPROMISE

Positive Attitude

High Level of Class Participation

No Plagiarism, Cheating …

No Change in Deadlines

No Usage of Mobile / Other Devices

…


Scenario 1

ABC Pvt Ltd is a company with branches at

Karachi, Quetta, Peshawar and Lahore. The Sales

Manager wants quarterly sales report. Each

branch has a separate operational system.


Scenario 1 : ABC Pvt Ltd.

Karachi

Quetta

Peshawar

Lahore

Sales

ManagerSales per item type per branch

for first quarter.


Solution 1:ABC Pvt Ltd.

Extract sales information from each database.

Store the information in a common repository

at a single site.


Solution 1:ABC Pvt Ltd.

Karachi

Quetta

Peshawar

Lahore

Data

Warehouse

Sales

Manager

Query &

Analysis tools

Report


Scenario 2

One Stop Shopping Super Market has huge

operational database. Whenever Executives wants

some report, the OLTP system becomes slow and

data entry operators have to wait for some time.


Scenario 2 : One Stop Shopping

Operational

Database

Data Entry Operator

Data Entry Operator

ManagementWait

Report


Solution 2

Extract data needed for analysis fromoperational database.

Store it in warehouse.

Refresh warehouse at regular interval so that itcontains up to date information for analysis.

Warehouse will contain data with historicalperspective.


Solution 2

Operational

database

Data

Warehouse

Extract

data

Data Entry

Operator

Data Entry

Operator

Manager

Report

Transaction


Scenario 3

Cakes & Cookies is a small, new company. President

of the company wants his company should grow. He

needs information so that he can make correct

decisions.


Solution 3

Improve the quality of data before

loading it into the warehouse.

Perform data cleaning and

transformation before loading the data.

Use query analysis tools to support

adhoc queries.


Solution 3

Query and Analysis

toolPresident

Expansio

n

Improvemen

t

sales

time

Data

Warehouse


Case Study

AFCO Foods & Beverages is a new companywhich produces dairy, bread and meatproducts with production unit located atGujranwala.

There products are sold in all the region ofPakistan.

They have sales units at provincial HeadQuarters.

The President of the company wants salesinformation.


Sales Information

Report: The number of units sold.

113

Report: The number of units sold over time

January February March April

14 41 33 25


Sales Information

Report : The number of items sold for each product with

time

Jan Feb Mar Apr

Wheat Bread 6 17

Cheese 6 16 6 8

Swiss Rolls 8 25 21

Product


Sales Information

Report: The number of items sold in each City for each

product with time

Jan Feb Mar Apr

Karachi Wheat

Bread

3 10

Cheese 3 16 6

Swiss Rolls 4 16 6

Lahore Wheat

Bread

3 7

Cheese 3 8

Swiss Rolls 4 9 15

Product

Tim

e


Sales Information

Report: The number of items sold and income in each region for

each product with time.

Jan Feb Mar Apr

Rs U Rs U Rs U Rs U

Karachi Wheat Bread 7.44 3 24.80 10

Cheese 7.95 3 42.40 16 15.90 6

Swiss Rolls 7.32 4 29.98 16 10.98 6

Lahore Wheat Bread 7.44 3 17.36 7

Cheese 7.95 3 21.20 8

Swiss Rolls 7.32 4 16.47 9 27.45 15


Data Warehousing includes

Building Data Warehouse

Online Analysis/Analytical Processing (OLAP)

Presentation

RDBMS

Flat File

Presentation

Cleaning ,Selection &

Integration

Warehouse & OLAP serverClient

professional issues in computing...nuces, islamabad campus data warehousing - fall 2012 12 nuces,...

Documents