1

ITEC

45

0

INTRODUCTION OF WEEK 14 Assignment Discussion

Graded: 3-1-4 (Lab4: Query Optimization) Creating, reviewing, and interpretation are all important for this assignment. Analysis of execution plans: scan (full table, index range, index unique), order

of execution, join operations (hash, nested loop) Comparing the plans, not statistics Complexity of plan != efficiency of the query: I/O scans, sorting, joins are

expensive Understanding application query tuning

Due: 12-1 (Database Storage), 12-2 (Bulk Data Movement) Working: 13-1 (Research paper: database metadata management) Working: 3-1-5 (Final Project Write-up)

Review of previous week and module Metadata Management, Database Management Tools Oracle 10g Data Dictionary and Dynamic Performance Views

Overview of this week – Module 5 Data Warehouse Administration Course Summary Final Exam Discussion

20

11

Fall

ITEC

45

0

2

MODULE 5 Metadata, Tools, and Data WarehousingSection 4 Data Warehouse Administration

20

11

Fall

3

ITEC

45

0

DATA WAREHOUSE AND CHARACTERISTICS

A data warehouse is a subject-oriented, integrated, time-variant, non-volatile collection of data that is designed for query and analysis rather than for transaction processes.

Subject-oriented – data pertains to a particular subject instead of the many subjects pertinent to the company’s ongoing operations.

Integrated – consistent naming conventions, formats, encoding structures; from multiple data sources

Time-variant – data is identified with a particular time period, can study trends and changes

Non-updatable – data is stable in a data warehouse. Data loaded, and should not be removed.

20

11

Fall

4

ITEC

45

0COMPARISON OF DATABASE CHARACTERISTICS

20

11

Fall

5

ITEC

45

0

DATA WAREHOUSE AND BUSINESS INTELLIGENCE

A data warehouse usually contains historical data derived from transaction data and other sources.

It enables an organization to consolidate data.

It includes An extraction, transportation, transformation,

and loading (ETL) solution An online analytical processing (OLAP) engine Client analysis tools Reporting

20

11

Fall

6

ITEC

45

0

ANALYTICAL VS. TRANSACTION PROCESSING

Analytical processing – informational systems DSS – decision support system OLAP – online analytical processing Data mining – the process of mining or discovery of new

information in terms of patterns or rules from vast amounts of data

Transaction processing – operational system OLTP – online transaction processing

20

11

Fall

7

ITEC

45

0

DATA WAREHOUSE DESIGN

Star schema - data modeling technique used to map multidimensional decision support data into a relational database. It is excellent for ad-hoc queries, but bad for online transaction processing. It contains four components: Fact table Dimension tables Attributes Attribute hierarchies

Snowflake schema – a star schema in which the dimension tables have additional relationships

20

11

Fall

8

ITEC

45

0

STAR SCHEMA COMPONENTS

20

11

Fall

9

ITEC

45

0

STAR SCHEMA EXAMPLE

20

11

Fall

10

ITEC

45

0

DATA MOVEMENT – ETL PROCESS

ETL – Extract, Transform, and Load Capture – extract or obtaining a snapshot of a chosen

subset of the source data for loading into the data warehouse

Scrub or data cleansing – uses pattern recognition and AI techniques to upgrade data quality

Transform – convert data from format of operational system to format of data warehouse

Load – place transformed data into the warehouse and create indexes

20

11

Fall

11

ITEC

45

0

DATA WAREHOUSE PERFORMANCE

Perspectives of data warehouse performance Extract performance – how ETL process performs Data management – database design and data quality Query performance – OLAP tuning Server performance – hardware support

Automated summary tables Provide a proper set of aggregate information Commonly implement with materialized views or batch

operation tables

DBMS features to support data warehousing Materialized views – automatically creation of summaries Bitmap indexes – widely used in data warehousing, in

addition to B-tree Parallel execution – multiple processes work together

simultaneously to run a single SQL statement

20

11

Fall

ITEC

45

0

12

MODULE 5 Metadata, Tools, and Data WarehousingSection 5 DBA Rules of Thumb

20

11

Fall

13

ITEC

45

0

THE RULES OF THUMB

Personal DBA handbook Write down your own experience Categorize them in a searchable note or repository

Backup everything and plan for worst all the time Before making any changes, ensure that you can

recover from them

Automation and share your knowledge Create a systematic way to troubleshoot problems Create, reuse and share scripts Knowledge sharing will open many revenues for you

Next levels Understand the business, not just the technology Keep up-to-date on technology

20

11

Fall

14

ITEC

45

0

COURSE SUMMARY (YOUR LEARNING)

DBA Roles and Responsibilities DBMS Architecture, Physical and Logical

Structures DBMS Installation and Database Creation Database Connectivity and Network

Components Database Security and Audit Capability Database Backup and Recovery Database Monitoring, DBMS System Tuning,

Physical Configuration Optimization SQL Query Coding and Tuning, Data

Loading Database Metadata, Data Dictionary Data Warehouse Characteristics and

Overview

20

11

Fall

15

ITEC

45

0

FINAL EXAM Midterm coverage (30%) Backup choices, recover mechanisms and high availability

features Performance influential factors, Database performance tuning Optimizer overview and optimizer influential factors Oracle query optimizer processing, statistics collection,

execution plan Oracle physical and logical database structures Space management, RAID technology The load utility, data pump export and import DBMS Metadata, metadata type Oracle data dictionary, dynamic performance views Data warehouse, characteristic differences vs. operational

database, analytic (OLAP) vs. transactional processing (OLTP) Data warehouse database design (star schema), ETL

20

11

Fall

16

ITEC

45

0

SCHEDULE REMINDER ONE MORE TIME

Final exam can be taken between Thursday, Dec. 8 and the week after Wednesday, Dec. 14.

The final exam must be completed on or before Wednesday of Week 15, not Sunday! Check with your proctor or test center.

20

11

Fall

All assignments are due by Sunday, December 11, and no late assignments will be accepted after the date.

Please review your grade book, and let me know any missing grades right way.

17

ITEC

45

0

THANK YOU AND GOOD LUCK

20

11

Fall