design and implementation of an enterprise data warehouse

8/8/2019 Design and Implementation of an Enterprise Data Warehouse

1/91

Marquee University

e-Publications@Marquee

Master's Teses (2009 -) Dissertations, Teses, and Professional Projects

Design and Implementation of an Enterprise Data Warehouse

Edward M. Leonard Marquee University

Recommended CitationLeonard, Edward M., "Design and Implementation of an Enterprise Data Warehouse" (2011). Master's Teses (2009 -). Paper 119.hp://epublications.marquee.edu/theses_open/119

http://epublications.marquette.edu/http://epublications.marquette.edu/theses_openhttp://epublications.marquette.edu/diss_theseshttp://epublications.marquette.edu/diss_theseshttp://epublications.marquette.edu/theses_openhttp://epublications.marquette.edu/


2/91

DESIGN AND IMPLEMENTATION

OF AN ENTERPRISE DATA AREHOUSE

B

E M. L, B.S.

A T F G S,

M U,

P F R D M S

M,

D 2011


3/91

ABSTRACT

DESIGN AND IMPLEMTATION

OF AN ENTERPRISE DATA AREHOUSE

E M. L, B.S.

M U, 2011

T

. N ,

, , . H,

,

. T

,

.

T T , , ,

,

. T T

. T T

. I ,

. F,

, .

T

. A

E D

E, T L (ETL)

. S,

, . A

,

.


4/91

E M. L, B.S.

I

. I , J O

.

I M U

.

F, I

. D. P M

. A ,

. D. H, ,

M S C .

D. K,

.


5/91

ACKNOLEDGMENTS ......................................................................................................................

TABLE OF CONTENTS........................................................................................................................

LIST OF TABLES ................................................................................................................................

LIST OF FIGURES ...............................................................................................................................

CHAPTER 1: I ................................................................................................................ 1

P ....................................................................................................................................... 1

E .................................................................................................................................. 2

M ................................................................................................................................... 3

Q............................................................................................................................... 6

CHAPTER 2: B ................................................................................................................. 8

CHAPTER 3: B I ................................................................................................. 21

R ................................................................................................................................... 22

CHAPTER 4: S A .................................................................................................. 28

D L ............................................................................................................ 29

ETL L P ................................................................................................................ 31

M ................................................................................................................................... 36

D M .............................................................................................................. 37

S S ........................................................................................................................... 38

D M ................................................................................................................................. 43

S S .................................................................................................................... 47

ETL A .......................................................................................................................... 48

CHAPTER 5: I ........................................................................................................ 50

E D P ..................................................................................... 50

ETL C ....................................................................................................................... 53

S S S ............................................................................................................. 54

E S S .......................................................................................................... 59

R................................................................................................................................... 63


6/91

CHAPTER 6: P E ........................................................................................... 71

O ................................................................................................................................... 71

T P ............................................................................................................................... 73

T S 1: S A T ....................................................................................... 73

T S 2: S A R ...................................................................................... 74

T S 3: S ............................................................................................. 76

CHAPTER 7: CONCLUSION ............................................................................................................. 77

BIBLIOGRAPH .............................................................................................................................. 79

A A .................................................................................................................................... 81

T T G ........................................................................................................... 81


7/91

T 1: T S 1 R ( M) ................................................................................... 74

T 2: T S 2 R ( M) ................................................................................... 75

T 3: T S 3 R ( M) ................................................................................... 76


8/91

F 1: C E D . ...................................................................... 10

F 2: T H A D . ....................................................... 29

F 3: T S S ............................................................................................................ 39

F 4: A D T ......................................................................................................... 40

F 5: F T E ........................................................................................................ 42

F 6: T HR S O E E ................................................................ 51

F 7: T S S M SQL S ......................................................... 52

F 8: T C ETL P. .......................................................................................... 54

F 9: T S C B S S S. ........................................ 55

F 10: A D E, T L. ........................................................... 56

F 11: T B FACTS T. ........................................................................ 58

F 12: T L T E L J. ....................................................... 59

F 14: T E S E P A D T. ........................................ 62

F 15: T S D S. .................................................................... 63

F 16: T R R D I S. ..................................................... 65

F 17: T R R I S. ..................................................................... 66

F 18: T R R G R. ................................................................ 67

F 19: T R R G R. ................................................................... 68

F 20: T R R S R B T. .............................................. 69

F 21: T R R S R. ................................................................... 70

F 22: G R S A Q. ....................................................................... 74

F 23: G R S A R Q. ........................................................ 75

F 24: G R S A C Q. ............................................... 76


9/91

1

1

T E D

. T

,

.

T E D ,

,

.

, , , , , ,

,

. M ,

. A

,

;

, . C,

. T, ,

, .


10/91

2

O E D

. S ,

. F, ,

M E ,

. A

, ,

. R

,

.

A B I

. T

, OLAP, O

A P. T

. O ,

, .

F,


11/91

3

. T

B I .

T

E D . A

, / ,

.

T

. C

, . A

, .

U ,

, .

I ,

. N,

. R

,

. F,

, .


12/91

4

A

. T

, ,

.

O ,

. D

E, P P,

OLAP, K P I (KPI). B ,

, ,

, .

B

, I T (IT) ,

. R

, ,

/ . F, IT

.

F IT ,

E D . M


13/91

5

. E

, ,

, . B

E D ,

.

P,

. A

. I ,

, . A

. A

,

, ,

.

T E D

. R ,

. T

.

N , ,


14/91

6

. H

. T E

, . T

. S

SO , .

I

. T

,

, . U

, . T

,

. T ,

, . O

E D

.

T E D

,


15/91

7

. F,

SQL S 2008 E

D . S, HR O

. T

. F ,

. E D ,

,

.


16/91

8

2

A ,

E D . B ,

.

, ,

E D

. O, ,

. E

,

.

( I T ),

.

S

, . F

,

,

. T ,

E D

. A ,


17/91

9

,

. D

, E D

. T

U C B, E D

,

(

I, 3).

E D ,

. I ,

,

. N,

,

. T

,

. A

. T

, ,

. A E D

( I, 6).


18/91

10

T F 1

E D .

1: .

E D ,

,

. T E D

,

, . T E D

. A


19/91

11

. T E D

. S

,

.

A,

,

.

B ,

. T

. T

. A

, . T

,

E D . I

,

.

, . T

: F,


20/91

12

? N,

? F, ?

B ,

. T E D

, , ,

, .

H ,

. I

,

. I ,

,

, . T

E D .

ETL , E D

. F,

.


21/91

13

T E D

. ,

,

. H, ,

. F,

. T

OLAP,

, .

E D ,

. T

,

, ,

P M I. E

, .

I

,

. I IT

, .


22/91

14

O E D ,

.

E : S

. (K, L, 16). B

,

. A

,

. F, E

(K, L, 17). N ,

. I

; , .

T

E D ,

. I , O

D/BI

(K, L, 17).

T

,

. T


23/91

15

, . A ,

D/BI

. (K, L, 17) T , ,

,

.

E D , I T

.

, IT

.

, ,

IT

P M ,

.

T

. T


24/91

16

. T I D/BI

D/BI (K, L, 35),

.

T

. F,

.

O ,

,

. T

, ,

. T

. R K ,

,

. H , , ,

(K, L, 65). T ,

. U

,

, .

B ,


25/91

17

D/BI (K,

L, 58),

.

S ,

,

,

. B ,

. O

,

. R ,

,

.

T

, . A

(S) ETE D Q,

BI .

T BI

. T


26/91

18

BI , . T

,

(S, 1). F, : 1) H

? 2) ? 3) H BI

? A, , 4) H (S, 1)?

I

,

.

B

,

. I (MK) D Q N

D,

. H 11

, : 1) A , 2)

I , 3) U

, . T

,

. H

, , , ,

. (MK, 1) T

.


27/91

19

C ,

.

C

, : I

,

, ? D , ,

. H

MK ,

, . A

ETL ,

.

B E D ,

. T

, ,

. B ,

. A


28/91

20

. U

. T

. T

,

.


29/91

21

3

T E D

,

. A

,

. N,

,

. F, ETL (E, T L)

. S ETL

SQL S I S, OBIEE O

I, . T

. T

,

. T

,

. N,

,

. H,

,

, , .


30/91

22

N , ,

. T ,

IT . S

. I E D

, BI

. I , ,

, , (S,

1). A ,

, ,

, .

T

. T

. T

SQL O. A

. T

. I ,

? F, M O

? T SAP


31/91

23

B O, MS, IBM C, A, J P. T

. , , , .

R : ,

. ,

(R, 1). I ,

,

, .

,

,

. T H C B I, 2010

. T

, ,

. B ,

BI . T

, . T

BI

. T P , BI

. F

, .

I M Q B I P G R,

BI


32/91

24

BI . T , , .

T

. T ,

.

T BI

. T

, SAP, M O

.

D

,

. T

. (K, L, 479). N

,

,

. I ,

SQL

.

, I

T

.


33/91

25

T : T

(, 31). T

. S

SAP C R B

M. T M , A , R B,

, SQL S 2008 (, 30). M C

B O E , M

SQL S R S,

. T IT

,

.

O

,

. T

,

(K, L, 491). A

, , , . O

,

. A ,


34/91

26

. I

E D , F S U

: I . P ,

. A

(S, 3).

A

OLAP (O A P) . OLAP

.

D

(K, L, 479). A M SQL A S,

. T

,

, KPI. T P P

E 2010 ,

, ,

(, 31). G

.

A

, .

T


35/91

27

. B

D/BI ,

,

. I , K

, , , ,

(K, L, 500). B ,

.


36/91

28

4

T E D

. T R K

K G

. T K

. A

, , .

T ETL E, T

L ( ). O

,

.


37/91

29

2: .

T

. A F 2, ,

, , ,

(K, T, 7). T

. N,

,

ETL

. T

, .

F,

.

T

. G, ETL, ,

,

. F

ETL , GUI

L UNI .

,


38/91

30

. D ,

S L HPUNI. B ,

,

.

T

. T

, CPU

. B

,

. K : T

, 80 D/BI

(K, L, 165). A ,

,

. , ,

,

.

F, . A

,

. I ,


39/91

31

. H, OLAP ,

. F

, T , ,

, , , (K, L, 167).

R,

. C

, SAP B O , UNI

. T .

O ,

.

A ETL

,

. D S

O D S,

, ( )

. S F 1,

. A ,

, , ETL

. T,

. O


40/91

32

,

ETL . O

,

ETL .

T K

ETL . H

. A K

, I ,

(K, T, 8). T ,

,

ETL

. F,

,

.

O

. I , ,

D , ,

, O

( D M, 39). S

,

. O


41/91

33

E D ,

. I ,

ETL

,

ETL.

A

. T

. D

(R, 1), IT

. O ,

,

ETL ETL

. T

,

.

,

.


42/91

34

,

. B

,

. A

,

. T, D

,

(R, 1). N

, S O,

, HIPPA.

T ,

. K ,

, ( ,

, , ),

, DE , (K, T, 8).

H

ETL . T

, ,

. , SQL S I

S ETL ,


43/91

35

, F L. T F L

; S T

(, 94). T

ETL,

,

.

T

. A K

,

ETL. N,

ETL

. A ,

,

. , ETL

. O

,

.


44/91

36

A

, . A

T.T. S.S. A

, , . T

M : M ;

,

, ,

( A, 671). T

,

( A,

671). M ,

. N,

,

, . F, ,

.

S ETL

. A J L B

ETL . A


45/91

37

ETL , . T

ETL ,

, ETL

. O ETL , D ,

. T ETL (L

, 2568). T ETL ,

, . T

ETL. I , ETL

ETL .

T

D M,

. K : D

D/BI . P

(K, L, 233). T

DE

. A ETL

,

.

F,


46/91

38

,

.

T ,

. T

,

(A, 10). T ( F 3 ),

, . T

,

. O ,

, . T

. S

,

, (A, 39). T

.


47/91

39

3:

T

. T

,


48/91

40

. T

, , ,

(K, L, 242). T , ,

, ,

. T

. I ,

, , . F 4

. T ETL

, . T

, ID

, I SQL S. T

ID ,

ETL.

4:


49/91

41

T

.

T

. T

ETL . T

(K, L,

257). T

. F, ETL

, , ,

. C

.

T ,

, .

T .

I ,

(K, T, 18). T

. I ,


50/91

42

F 5, FACTE

, . I

,

.

5:

I ,

(K, L, 238). U

,

. T

FACTE ,

,

, . I ,

. F,

S F,

.


51/91

43

,

, . T

,

. T

; (K,

L, 346). H

.

S ,

. I

, .

T

,

. T

. F,

,

. T ,

I IT

. S ,


52/91

44

, .

S,

(). N, ,

,

.

R ,

,

. B ,

K ,

1990,

(K, L, 248). T

.

H, IT ,

, ,

(K, 248). S

. R K , ,

E D

.

I ,

, ,


53/91

45

. I

.

T , E D ,

,

. H,

E D

,

(S, 8). D

.

O ,

.

, T

. A

( I D M, 1). I

, E D

,

. I ,

.


54/91

46

I E D ,

,

. B ,

, . E

( I

D M, 1). T , ,

,

. L,

I

. C,

. T,

, ,

.

T

. F,

,

, ETL

, . S, ,

,

. I ,


55/91

47

(K, T, 217). H

,

.

O ,

. ,

,

.

T ,

, . T

(A, 157).

,

.

M ;

. I

, I .

,

.


56/91

48

T ,

. H,

. T

. I ,

(U, 37).

T

. T

. S

ETL .

T

. T

. T

(S, C A), (, , )

/ (E, M, S P

(U, 39). T

. I ,

,


57/91

49

. N, , ,

. F,

, , .

T

. T E

, E

(U, 40). T

C# ,

. I

,

ETL. S,

ETL ,

. O E D

, .


58/91

50

5

T E D

. T

B I SQL S SQL S ,

.

T E D

. T D

,

.

A ,

, .

T ,

, . T

,

. T

. T

. A ,


59/91

51

. I

E D ,

. T SQL S 2008 R2 , SQL

S I S ETL SQL S R S

.

T HR , O E

10.2 (E E) . A F 6, HR

. HR , ,

, , , (B, 19). T

, .

6:


60/91

52

T

S, ,

. T , ,

, . T

M SQL S 2008R2 ,

, . T

M D N,

. I F 7,

:

7:


61/91

53

T

, , HR

. T ,

ID . S

, . T F

F ID D ID

. F

, . T

. H,

,

, .

B F 8, ETL SQL S I S,

S D . E S C,

. T

.


62/91

54

.

8: .

T ETL F 9

,

.


63/91

55

9: .

T D F ETL .

, , . S

, ETL . I

ETL,

. T

SQL. I ,

. F ,

, .


64/91

56

10: , .

T F 10

SQL,

. H :

DIMC

SELECT CourseID, Title, Credits, DepartmentIDFROM COURSE

DIMI ( )

SELECT *FROM DBO.PERSON

WHERE HIREDATE IS NOT NULL


65/91

57

DIMD

SELECT *FROM DBO.DEPARTMENT

DIMS

SELECT *FROM DBO.PERSONWHERE HIREDATE IS NULL

T . I

,

.

O ,

. T

. I ,

, SQL

. I , ID

.


66/91

58

11: .

T F 11

SQL,

. H :

M E

SELECT S.*, PersonIDFROM dbo.StudentGrade SLeft Join dbo.CourseInstructor C ON C.CourseID = S.CourseID

T L. T

F 12

,

.


67/91

59

12: .

T

E D . T

O HR

.


68/91

60

13: .

A ,

. T F 13,

, . I

,

. F , I

SQL S I S O ,

SQL S . N,

. ,


69/91

61

I

O SQL S D . T

, ,

,

. A

.

M

SSIS SQL. A SSIS TSQL,

O . H, ETL

SQL , . I ,

, I F 14,

S * .


70/91

62

14: .

T

ID . T

. T

. T

E H J


71/91

63

H E

E . O

E D

. S,

.

T F 15:

15: .

N ,

E D . T


72/91

64

SQL S O HR

. T , , HR

. U ,

HR . T

PID P . T

. A , DI DIME

E ID,

. I , I

. F , S D, . B

DIMI T DIME HR ,

FACTE . T

.

T I

E D ,

. T , SQL S I S (SSRS),

M BI ,

. B

, . T

:

SELECTFACT_Employee.EmployeeFactID,FACT_Employee.EmployeeDimID AS [FACT_Employee

EmployeeDimID],FACT_Employee.JobDimID


73/91

65

,DIM_Employee.EmployeeDimID AS [DIM_EmployeeEmployeeDimID],DIM_Employee.EMPLOYEE_ID,DIM_Employee.SALARY AS [DIM_Employee SALARY],DIM_Instructor.InstructorDimID,DIM_Instructor.PersonID

,DIM_Instructor.LastName,DIM_Instructor.FirstName,FACT_Employee.SALARY AS [FACT_Employee SALARY]

FROMFACT_EmployeeINNER JOIN DIM_Employee

ON FACT_Employee.EmployeeDimID =DIM_Employee.EmployeeDimIDINNER JOIN DIM_Instructor

ON DIM_Instructor.EmployeeId = DIM_Employee.EMPLOYEE_ID

16: .


74/91

66

17: .

T I

. T , SQL S I S (SSRS)

D . B

,

. T :

SELECT E.Grade, I.LastName as Instructor, C.CourseID,C.Title, C.Credits, S.LastName, S.FirstName, D.NameFROM FACT_ENROLLMENT E

INNER JOIN DIM_Instructor I ON E.InstructorDimid =I.InstructorDimIDINNER JOIN DIM_Course C ON C.CourseDimID = E.CourseDimIDINNER JOIN DIM_Student S ON E.StudentDimId = S.StudentDimId


75/91

67

INNER JOIN Dim_Department D ON E.DepartmentDimID =D.DepartmentDimID

18: .


76/91

68

19: .

T I

. T ,

SQL S I S (SSRS)

D . S FACTE

, DIME

. B ,

. T :


77/91

69

SELECT FE.SALARY, JOB_TITLE, EMPLOYEE_ID, LAST_NAMEFROM dbo.FACT_Employee FEINNER JOIN DIM_Employee E ON E.EmployeeDimID =FE.EmployeeDimID

20: .


78/91

70

21: .

T

HR . M ,

. S

.


79/91

71

6

T E D ,

. T

, .

I ,

.

T

. I ,

. D

, I ETL

. T

. T

, .

T

. T

. T ,

, ,


80/91

72

, (S . 1). I

, ,

, . I ,

ETL ,

, ,

.

F ,

,

. H

, ETL . B

,

, DBMS

, (S . 7). T

.

I ETL

SQL ,

. T

ETL. S

,


81/91

73

, . H,

, ,

.

I , I

. T O

SQL S , O

. T , , O SQL

T SQL TSQL SQL S . T O

SQLP, . C, I SQLCMD

SQL

.

1

T , 100

100,000 10 . T

, . A

F 22,

.


82/91

74

DM D

S TOP 100 00:06.9 00:00.0 00:06.9

S TOP 1000 00:07.9 00:00.2 00:07.7

S TOP 10000 00:18.9 00:01.7 00:17.2

S TOP

10000002:14.8 00:17.8 01:57.015

S * 02:42.0 00:35.5 02:06.5

1: 1 ( )

22: .

2

T ,

1 100,000 10 . T ,

. A


83/91

75

F 23,

. O

,

.

DM D

SELECT 1 R 00:03.5 00:00.2 00:03.3

SELECT 100 R 00:03.6 00:00.2 00:03.4

SELECT 1000 R 00:04.8 00:00.4 00:04.4

SELECT 10000

R00:19.9 00:02.0 00:17.9

SELECT 100000R 02:38.7 00:57.3 01:41.357

2: 2 ( )

23: .


84/91

76

3

T

. A

,

. R ,

, F 24.

DM D

SELECT * HERE U 100 R 00:00.6 00:00.2 00:00.4

SELECT * HERE A 1000 R 00:05.0 00:00.5 00:04.4

SELECT * HERE A T 10000 R. 00:29.1 00:14.9 00:14.2

SELECT * HERE F 100000 R 04:02.0 01:06.4 02:55.7

3: 3 ( )

24: .


85/91

77

7

T E D

,

. I ,

. T IT

.

T E D

,

. B

,

. ,

,

.

T ,

E D . B

, E D ,

ETL

. ,

.


86/91

78

,

.

S

, E D

.


87/91

79

A, C, . N : MG H, 2010.

B , C. E . O D S S 10 R 2 (10.2). .J 2005. O C. 27 M 2011.

://..///B1930601/.102/14198.

B, A. H C B I, 2010. . 16 A

2010. G, I. 10 A 2011.

C S S D.

://..///399731.. M C, .. . 26

M 2011.

E D I ED P P. 23 F. 07.

U C B. . 7 M 2011.

K, R. E . . S E. N : C

P, 2002.

. . S E. N : P, I.,

2008.

L, J; , B; "ETL ,"

(), 2010 ,

.6, ., .25672569, 1012 A. 2010 : 10.1109/FSKD.2010.5569836

://..//.?=&=5569836&=5569072

MK , . D Q N D. . N 2010. .

20 O 2011. ://..//207/

100191121..

R T S D . 1.. .. . 20 O 2011.

://.1.//..

R, D. D A D D S A. . 22 J 2010. . 20 O 2011.

://..//./3888696/DA

DDSA..

S, R. E . M Q B I P. . 27

J 2011. G, I. 25 A 2011.


88/91

80

G. S, S. S, "D E P RDBMS," , . 3, .3013,

34 A H I C S S ( HICSS34) 3, 2001

S, D. F S U T R B I.

. A ,2010.

S, P. E E D Q. . . 20 O 2011.

://../2008/10//.

S, A. 0 . N : C P, 1998.

T D S A. . , .. . 20 O 2011.

://.//..

U, M.; A, S.; F, S.; , "D OLAP ,"

(), 2010 , ., ., .3543,

58 J 2010 : 10.1109/ICDIM.2010.5664622

://..//.?=&=5664622&=5662242

, E. E . ( 70445) 2005.

R: M P, 2008.

A D. .. , .. . 20 O 2011.

://...///..

, T.T.; A, S.S.; , "M B S D E U D

," 200 , ., ., .670673, 1517 M 2009 : 10.1109/ICSPS.2009.101

://..//.?=&=5166871&=5166728

, M. E . B P D SQL S 2008 R2.

. D 2010. M C. 24 J. 2011.

://..///567302.


89/91

81

A A () (K, T, 391).

B (BI) A

(K, T, 393).

C N

(OLAP) , ,

, (K, T, 395).

D M A (K,

T, 396).

D M A , ,

(K, T, 397).

D T

,

(K, T, 397).

D A , ,

(K, T, 398).

D A ,

, (K, T,

398).

D T A

(K, T, 399).

E (ED) T

(K, T, 400).

ETL S

(K, T, 401).

E M L (ML) A HTML

. ML (K,

T, 402).


90/91

82

F T I ( ),

,

(K, T, 402).

F K A

. I ,

(K, T, 402).

I A

(K, T, 404).

OLAP OLAP

(K, T, 408).

O D S ( "ODS") A

(K, T, 408).

P K A

(K, T, 410).

RDBMS D

SQL. U

(K, T, 411). O O C RDBMS

SQL S M C.

S (SCD) T

. A 1 SCD

. A 2 SCD

. A 3 SCD

(K, T, 413).

S S A ,

. I

, ,

,

(K, T, 413).

SQL S Q L,

(K, T, 414).


91/91

83

S S T

,

(K, T, 414).

S K I

. I ,

. I ,

, (K, T,

414).

T C () () (K, T, 414).

SQL

SELECT (K, T, 417).

design and implementation of an enterprise data warehouse

Documents