ppt - data warehouses and function points - ifpug proceedings/ifpug-2004... · 3 5 terms • source...

14
1 1 2 Agenda • Terminology • Structure Counting Transactions Counting Data Stores

Upload: lydan

Post on 16-Apr-2018

227 views

Category:

Documents


8 download

TRANSCRIPT

Page 1: PPT - Data Warehouses and Function Points - IFPUG Proceedings/IFPUG-2004... · 3 5 Terms • Source system • Staging Area • eDW • Data Mart • ETL 6 Boundaries rules (IFPUG

1

1

���������������� �

�������������

����������

2

Agenda

• Terminology• Structure• Counting Transactions• Counting Data Stores

Page 2: PPT - Data Warehouses and Function Points - IFPUG Proceedings/IFPUG-2004... · 3 5 Terms • Source system • Staging Area • eDW • Data Mart • ETL 6 Boundaries rules (IFPUG

2

3

What is hard?

• New set of terms• Similar data in different places• Getting ‘sucked’ into the physical world

4

DW Structure

Page 3: PPT - Data Warehouses and Function Points - IFPUG Proceedings/IFPUG-2004... · 3 5 Terms • Source system • Staging Area • eDW • Data Mart • ETL 6 Boundaries rules (IFPUG

3

5

Terms

• Source system• Staging Area• eDW• Data Mart• ETL

6

Boundaries rules (IFPUG CPM 4.2)

• The boundary is determined based on the user’s view. The focus is on what the user can understand and describe.

• The boundary between related applications is based on separate functional areas as seen by the user, not on technical considerations.

• The initial boundary already established for the application or applications being modified is not influenced by the counting scope.

Page 4: PPT - Data Warehouses and Function Points - IFPUG Proceedings/IFPUG-2004... · 3 5 Terms • Source system • Staging Area • eDW • Data Mart • ETL 6 Boundaries rules (IFPUG

4

7

Boundaries rules

• User view is key• IFPUG manual also lists hints to validate

against during the boundary discussion.– Look at how groups of data are being

maintained– Look at associated measurement data, such as

effort, cost, and defects. The boundaries for function points and other measurement data should be the same.

8

Boundary example

Page 5: PPT - Data Warehouses and Function Points - IFPUG Proceedings/IFPUG-2004... · 3 5 Terms • Source system • Staging Area • eDW • Data Mart • ETL 6 Boundaries rules (IFPUG

5

9

FPA

• Elementary Process importance• Processing logic

10

ETL and transactions

• Extract

• Transformation

• Load

Page 6: PPT - Data Warehouses and Function Points - IFPUG Proceedings/IFPUG-2004... · 3 5 Terms • Source system • Staging Area • eDW • Data Mart • ETL 6 Boundaries rules (IFPUG

6

11

Transaction counting

A

12

Transactions

• ETL Map– Overview information– Source/Target– Required transformations– High level process overview– Process description– Source to Target Field Matrix

Page 7: PPT - Data Warehouses and Function Points - IFPUG Proceedings/IFPUG-2004... · 3 5 Terms • Source system • Staging Area • eDW • Data Mart • ETL 6 Boundaries rules (IFPUG

7

13

High level process overview

INV 8.17exp_Determine_Whether_Update_Or_Insert

Lookups

INV 8.1sq_Transform_STG_Invc_Detail_To_EDW_Invoice_Li

ne_Item

Error_Log

INV 8.2exp_Validation_and_Get_Session_R

un_ID

INV 8.3lkp_Session_Run_

ID

disconnectedlookup

EDW_Invoice_Line_Item

INV 8.7lkp_EDW_Except_Prc_Rsn_Code_F

orException_Price_

Reason_C

INV 8.10lkp_EDW_Shipping_Platform_Code_For_Shipg_Pltfrm_

C

INV 8.6lkp_EDW_Free_Prod_Code_For_Free_Product_Type_

C

INV 8.9lkp_Spcfd_Promo_Code_For_Specified_Promotion_C

INV 8.11lkp_Invoice_Adj_Type_Code_For_Inv

_Adj_Typ_N

INV 8.4lkp_EDW_Invoice_Header_For_Inv_N_Inv_Adj_N_Prcs

_Ts

INV 8.5lkp_EDW_Packaged_Good_ID_For_Mand_ID_N_Upc_

C_Upc_Sufx_C

INV 8.8lkp_EDW_Invoice_Reason_Code_Or_Line_Maint_Rsn_Code_For_Rsn_

C

INV 8.13sq_Invc_Detail_Err

ors

INV 8.14jnr_STG_And_

_Invc_Detail_Errors

INV 8.15fil_No_Error_Or_Non_Critical_Error_For_EDW_Invoice

_Line_Item

INV 8.16lkp_EDW_Invoice

_Line_Item

INV 8.19fil_Failed_Validation_For_EDW_Invoi

ce_Line_Item

INV 8.20nrm_Failed_Validation_For_EDW_Invoice_Line_Item

INV 8.22fil_Remove_Zero_Error_Msg_Id_For_EDW_Invoice_Li

ne_Item

INV 8.18upd_EDW_Invoice

_Line_Item

INV 8.21seq_Err_Seq_ID_For_Invoice_Line_

Item

INV 8.12lkp_EDW_Accounting_Time_For_Fis

cal_Period

INV8.23exp_Choose_Correct_Record_To

_Use

Invc_Detail_Errors

EDW_Invoice_Header

INV 8.24agg_Find_Unique_Invoice_Header_I

Ds

INV 8.25upd_Header_Updt

_Date

Possible data stores

Possible two E.P.

14

Target Table Target Column Datatype Source Table Source Column Datatype Data Issues/QualityEDW_Invoice_Line_Item Invoice_Line_Item_ID Number(1

0)Sequence Generator EDW Primary key

EDW_Invoice_Line_Item Invoice_N Varchar2(7)

STG_Invc_Detail Inv_N Varchar2(7)

Source/STG Primary key

EDW_Invoice_Line_Item Invoice_Adjustment_N Number(5)

STG_Invc_Detail Inv_Adj_N Number(5)

Source/STG Primary key

EDW_Invoice_Line_Item Invoice_Line_N Number(5)

STG_Invc_Detail Inv_Lin_N Number(5)

Source/STG Primary key

EDW_Invoice_Line_Item Process_Ts Date STG_Invc_Detail Prcs_Ts Date Source/STG Primary keyEDW_Invoice_Line_Item Invoice_Header_ID Number(1

0)EDW_Invoice_Header

Invoice_Header_ID Number(10)

EDW_Invoice_Line_Item Packaged_Good_ID Number(10)

EDW_Packaged_Good

Packaged_Good_ID Number(10)Number(2)

Number(4)

STG_Invc_Detail

Ord_N, Entry_D

EDW_Order_Header

Shipment_Id

EDW_Shipment

Load_Ord_N

STG_Invc_Detail Ord_N, Entry_D

EDW_Order_Header

Order_Type_C

STG_Invc_Detail Ord_N, Entry_D

EDW_Invoice_Line_Item

Fiscal_Period Number(6)

STG_Invc_Detail, EDW_Month

Use Actg_Mth_N + Actg_Yr_N to look up Fiscal_Period

EDW_Invoice_Line_Item

Load_Order_N Varchar2(7)

Varchar2(7

Look up shipment_id on EDW_Order_Header (by ord_n, entry_d ). Look up load_ord_n on EDW_Shipment (by shipment_id)

EDW_Invoice_Line_Item Order_Type_C Varchar2(2)

Varchar2(2)

Look up order_type_c on EDW_Order_Header (by ord_n, entry_d)

EDW_Invoice_Line_Item Submit_Method_C Varchar2(2)

Varchar2(2)

Look up submit_method_c on EDW_Order_Header (by ord_n, entry_d)

Derived field

Invoice ETL mapSource to Target Field Matrix

Page 8: PPT - Data Warehouses and Function Points - IFPUG Proceedings/IFPUG-2004... · 3 5 Terms • Source system • Staging Area • eDW • Data Mart • ETL 6 Boundaries rules (IFPUG

8

15

DET identificationTarget Table Target Column Datatype Source Table Source Column Datatype

EDW_Invoice_Line_Item Invoice_Line_Item_ID Number(10)

Sequence Generator

EDW_Invoice_Line_Item Invoice_N Varchar2(7)

STG_Invc_Detail Inv_N Varchar2(7)

EDW_Invoice_Line_Item Invoice_Adjustment_N Number(5)

STG_Invc_Detail Inv_Adj_N Number(5)

EDW_Invoice_Line_Item Invoice_Line_N Number(5)

STG_Invc_Detail Inv_Lin_N Number(5)

EDW_Invoice_Line_Item Process_Ts Date STG_Invc_Detail Prcs_Ts DateEDW_Invoice_Line_Item Invoice_Header_ID Number(1

0)EDW_Invoice_Header

Invoice_Header_ID Number(10)

EDW_Invoice_Line_Item Packaged_Good_ID Number(10)

EDW_Packaged_Good

Packaged_Good_ID Number(10)

Number(2)

Number(4)

STG_Invc_Detail Ord_N, Entry_D

EDW_Order_Header

Shipment_Id

EDW_Shipment Load_Ord_N

STG_Invc_Detail Ord_N, Entry_D

EDW_Order_Header

Order_Type_C

STG_Invc_Detail Ord_N, Entry_DEDW_Invoice_Line_Item Submit_Method_C Varchar2(2)

Varchar2(2)

EDW_Invoice_Line_Item Order_Type_C Varchar2(2)

Varchar2(2)

EDW_Invoice_Line_Item Load_Order_N Varchar2(7) Varchar2(7

Use Actg_Mth_N + Actg_Yr_N to look up Fiscal_Period

EDW_Invoice_Line_Item Fiscal_Period Number(6) STG_Invc_Detail, EDW_Month

Target Table Target Column Datatype Source Table Source Column DatatypeEDW_Invoice_Line_Item Invoice_Line_Item_ID Number(1

0)Sequence Generator

EDW_Invoice_Line_Item Invoice_N Varchar2(7)

STG_Invc_Detail Inv_N Varchar2(7)

EDW_Invoice_Line_Item Invoice_Adjustment_N Number(5)

STG_Invc_Detail Inv_Adj_N Number(5)

EDW_Invoice_Line_Item Invoice_Line_N Number(5)

STG_Invc_Detail Inv_Lin_N Number(5)

EDW_Invoice_Line_Item Process_Ts Date STG_Invc_Detail Prcs_Ts DateEDW_Invoice_Line_Item Invoice_Header_ID Number(1

0)EDW_Invoice_Header

Invoice_Header_ID Number(10)

EDW_Invoice_Line_Item Packaged_Good_ID Number(10)

EDW_Packaged_Good

Packaged_Good_ID Number(10)

Number(2)

Number(4)

STG_Invc_Detail Ord_N, Entry_D

EDW_Order_Header

Shipment_Id

EDW_Shipment Load_Ord_N

STG_Invc_Detail Ord_N, Entry_D

EDW_Order_Header

Order_Type_C

STG_Invc_Detail Ord_N, Entry_D

EDW_Invoice_Line_Item Fiscal_Period Number(6) STG_Invc_Detail, EDW_Month

Use Actg_Mth_N + Actg_Yr_N to look up Fiscal_Period

EDW_Invoice_Line_Item Load_Order_N Varchar2(7) Varchar2(7

EDW_Invoice_Line_Item Order_Type_C Varchar2(2)

Varchar2(2)

EDW_Invoice_Line_Item Submit_Method_C Varchar2(2)

Varchar2(2)

16

Invoice transaction

• What do we have so far?

Package_Good_item

EDW_Invoice_Line_Item

EDW_Invoice_Header

Invoice 4/9

Error_Log

Page 9: PPT - Data Warehouses and Function Points - IFPUG Proceedings/IFPUG-2004... · 3 5 Terms • Source system • Staging Area • eDW • Data Mart • ETL 6 Boundaries rules (IFPUG

9

17

Table types

• Facts• Dimensions• Aggregates• Metadata• Error Logs• Release or publish tables

18

Dimensional Model

• Fact tables • Dimensions

Page 10: PPT - Data Warehouses and Function Points - IFPUG Proceedings/IFPUG-2004... · 3 5 Terms • Source system • Staging Area • eDW • Data Mart • ETL 6 Boundaries rules (IFPUG

10

19

Fact tablesEDW_Invoice_Header

20

Dimension tables

Page 11: PPT - Data Warehouses and Function Points - IFPUG Proceedings/IFPUG-2004... · 3 5 Terms • Source system • Staging Area • eDW • Data Mart • ETL 6 Boundaries rules (IFPUG

11

21

Table counting

EDW_Invoice_Line_Item

EDW_Invoice_Header1. Identify the entities that should be considered for

counting.

2. Identify the user view (=business view) of the data grouping by investigating

a) How the data is accessed as a group by elementary processes within the application boundary

b) The relationships between the entities and their interdependency based on business rules

3. Classify each identified logical file as ILF or EIF

22

Data mart consideration boundary

Invoicing Warehousing

Data mart

Page 12: PPT - Data Warehouses and Function Points - IFPUG Proceedings/IFPUG-2004... · 3 5 Terms • Source system • Staging Area • eDW • Data Mart • ETL 6 Boundaries rules (IFPUG

12

23

Data mart dimensional model

EDW_Invoice_Line_Item

Time Dimension

Customer Dimension

Product Dimension Supply Point Dimension

Shipment DimensionOrder Dimension

24

Mapping matrixWarehouseFunction Point transaction Type FTRs DETs ETL mappings Comments

Inv 7Inv 8Inv 9Inv 10Inv 11Inv 13Inv 14Inv 15Iinv 16Inv 4

Data store RETs DETs CommentsInvoice headerInvoice Line item

Package Good Items EIF Package good items >51

Data MartFunction Point transaction Type FTRs DETs ETL mappings CommentsNot covered Data store RETs DETs CommentsTime ?Order ?Shipment ?Product ?Customer ?Supply Point ?Invoice ?

Load Invoice >3 >16

>51Invoice

EI

ILF

Page 13: PPT - Data Warehouses and Function Points - IFPUG Proceedings/IFPUG-2004... · 3 5 Terms • Source system • Staging Area • eDW • Data Mart • ETL 6 Boundaries rules (IFPUG

13

25

Summarize

• Application boundaries• Understand the terminology used for this

domain of applications

26

Questions

Page 14: PPT - Data Warehouses and Function Points - IFPUG Proceedings/IFPUG-2004... · 3 5 Terms • Source system • Staging Area • eDW • Data Mart • ETL 6 Boundaries rules (IFPUG

14

27

Sources/References• Kimball, Reeves, Ross, Thornthwaite, The Data Warehouse Lifecycle Toolkit:Expert

methods for Desighning, Developing, and Deplying Data Warehouses, WILEY 1998• IFPUG, FP-380 Function Point Counting Workshop: Counting Data Warehouse

Applicatoins• Santillo, Luca, Size & Estimation of Data Warehouse Systems, Data Processing

Organization• IFPUG, Function Point Counting Practices Manual Release 4.2