william yeoh

8/2/2019 William Yeoh

1/26

Developing an Integrated Metadata Framework in

Business Intelligence Environment

Dr. William Yeoh

School of Information Systems

Deakin University


2/26

BI Vendors


3/26

Business Intelligence System

1

Data

Repositories

Data Extraction,

Transform, Loading

Central

Data WarehouseData Marts Data Access and Analysis

Procurement

Cube

EnterpriseData

Warehouse

OperationalData Store

(ODS)

OtherData Mart

ProcurementData Mart

FinancialData Mart

Cube

Cube

Ad hocQuery

ExecutiveIS

Standard

Report

Data ModelingTool

ETL Tool

Staging

Area

CentralMetaData

Staging andETL process

BI Tools

ERP Suite

Other

Systems

OtherData Marts

Legacy

Systems

EAI Tool

Production

BI Portal

OLAP

Spread

Sheet

Metadata

Figure: Adapted from Griffin, J. (2003)


4/26

Some types of Data Quality Problems

Definition mismatches

InconsistentDatatypes

Inconsistent Null rules

Invalid keysInvalidValu

es

Orphans

Outofrang

e

Pattern exceptions

Potential

defaults

Duplica

tesRuleexceptions

Unused fields

Inconsistentdatatypes

Synonyms (Same object different names)

Homonyms (Different objects same name)

Spellingerrors

Dummyvalu

es

Absence of data

Multipurpose fields

Contradiction

Violation of business rules

Non-unique identifiers

Multipurpose fields

Cryptic data

Mismatched entries

Incomplete values

Missing values

Misfieldedvalues


5/26

Data + Business Rules = Information

QUALITY(Data + Business Rules) = Information

?


6/26

Case 1: A Large University

Object and Element Metadata Model


7/26

Research Participants

Business Intelligence Manager,

Project Officer,

Enterprise Data Warehouse Specialist,

Planning Officer,

Business Intelligence Analyst,

A Senior Analyst,

A Business Analyst, Key Business Users.


8/26

Metadata needs

?To provide consistency for descriptions and

definitions of the data in the BI environment;

?To provide an overall enterprise view;

?To solve a problem of misinterpretation of

some terms which could have different

meanings for staff with different roles; and

?To provide translation between technical and

business terms.


9/26

An Example of Glossary


10/26

Metadata for Object (reports & cubes)

Name of the object

Type of the object

Description

Time period

Scope

Usage

Primary audienceCritical notes

Source systems

History

Contact person

Report designer

Data Refresh date

Refresh frequency

Report run date


11/26

Metadata for Element (terms and

columns)

Name of the element

Business acronym

Primary system

Description

Type (measure or dimension?)

Places of use

History

Owner (maybe same as contact in

Obj)


12/26

Metadata interface(ASP 2.0)

Metadata database

(SQL Server 2005)

CognosViewer

Cognos

Content Store

ModelXML files

Metadata

import app(Windows

Form, VB.NET,

Cognos SDK)

Existing components of the BI environmentComponents of the metadata application

CognosJavaScript

files

ModifiedJavaScript

files

Architecture of the metadata application


13/26

Metadata Interface


14/26

Case 2: An Online Recruitment Firm

An Integrated Metadata Framework to

enhance BI Traceability


15/26


16/26

Research Participants

Chief Executive Officer,

Chief Operation Officer,

Chief Information Officer,

Chief Technology Officer,

Regional Process Improvement Manager,

Quality and Administration Manager,

Management Information Systems Manager.


17/26

The Metadata Framework: Business Intelligence

Product Map (BIP-Map)

BIP-Map


18/26

High-level Architecture of BIP-Map

BI Dashboard

BIP-Map Layer 1

(Business Process)

BIP-Map Layer 2

(Information Process)

BIP-Map Layer 3

(Metadata)


19/26

The First Layer of BIP-Map:

Business Process

? Based on Business Process Modelling Notation (BPMN).

? Users are able to easily identify the activities and

participants for the business processes.

? Changes can be implemented easily to improve the workflow

of business processes if necessary.

BI Dashboard

BIP-Map Layer 1

(Business Process)

BIP-Map Layer 2


BIP-Map Layer 3

(Metadata)


20/26

The Second Layer of BIP-Map:

Information Process

? Information Product Map (IP-Map)

? Users are able to visualise the information manufacturingprocesses of an information product.

? The way how data is being captured, processed and stored in

the systems can be easily understood.

? Enables changes to be implemented easily for improving the

workflow of information processes.

BI Dashboard

BIP-Map Layer 1

(Business Process)

BIP-Map Layer 2


BIP-Map Layer 3

(Metadata)


21/26

The Third Layer of BIP-Map:

Metadata Information

? Users can know in detail about how data is being validatedand

processed when it is being captured.

? Serves as a useful reference for database administrators.

? Helps users to identify the criteria used to generate a report

from the systems.

? Implement accountabilities to manage the quality of data at

enterprise level.

BI Dashboard

BIP-Map Layer 1

(Business Process)

BIP-Map Layer 2


BIP-Map Layer 3

(Metadata)


22/26

Demo of BIP-Map


23/26

Summary

? Visualise all business and information processes relating to the

manufacturing of BI products.

? Helps BI stakeholders to understand the data and the

information manufacturing processes.

? Enhances traceability of the BI product by clearly outlininglineage information.

? Allows technical personnel to understand the business processes

that relate to certain information.

? Business users are able to gain insights into the logic behind

any BI report.


24/26

BIP-Map


25/26

Acknowledgement

Yuriy Verbitskiy, Planning and Assurance Services,University of South Australia.

Tim Chee, Faculty of Information and Communication

Technology, University Tunku Abdul Rahman.

JobStreet.com Corporation

Research participants

References: Verbitskiy, Y. Yeoh, W & Koronios, A 2009, Metadata implementation

for a Business Intelligence environment, Proceedings of Annual

Conference of Australasian Association for Institutional Research,

Adelaide, Australia.

Chee, C.H., Yeoh, W. & Gao, S, 2011, Enhancing Business

Intelligence Traceability through an Integrated Metadata Framework,

Proceedings of 22nd Australasian Conference on Information

Systems, Sydney, Australia.


26/26

[email protected]

william yeoh

Documents