saranathan asuri mphasis

30
Asuri Saranathan

Upload: sarferazulhaque

Post on 09-Feb-2016

16 views

Category:

Documents


0 download

DESCRIPTION

Saranathan Asuri Mphasis

TRANSCRIPT

Page 1: Saranathan Asuri Mphasis

Asuri Saranathan

Page 2: Saranathan Asuri Mphasis

Agenda

Introduction Best Practices – Over View Deep Dive Conclusion Q & A

Page 3: Saranathan Asuri Mphasis

Introduction

Page 4: Saranathan Asuri Mphasis

Speaker

Holds Bachelor degree in Physics and Electrical and Electronics Engineering.

Over 26 years of Experience in Power System and Information Technology field.

Has built several large scale applications including online CRM for multinationals.

Has managed several Data Warehousing and BI projects for Direct Marketing, Manufacturing and Auto finance verticals.

Functioned as Solution Architect for Data warehousing and reporting projects.

ISO auditor Certified Bullet Proof Manager from CrestComm USA.

Page 5: Saranathan Asuri Mphasis

Best Practices- Overview

Page 6: Saranathan Asuri Mphasis

What are Best Practices?

Is it a technology? Is it application of a set of best tools

available in the market? Is it a Framework?

Page 7: Saranathan Asuri Mphasis

Best Practices Definition A framework of a set of processes

or method that exhibits achievement of specific results in a specific manner over a sustained period of time.

The framework should have certain characteristics in that they should be repeatable.

Page 8: Saranathan Asuri Mphasis

Do Best Practices Evolve? Yes they do.

Because of Innovation Changes in Technology Changes in Law or Governance

Structure. Expectations, Values , Knowledge or

other that makes the practice outdated or inappropriate.

Page 9: Saranathan Asuri Mphasis

Where can it be Applied?

Practically in all fields.

Page 10: Saranathan Asuri Mphasis

How do we apply Best Practices to Data Warehousing and Business Intelligence?

Page 11: Saranathan Asuri Mphasis

Data Warehouse - Definition In an elementary form , it is the

collection of key information that can be used by the business users to become more profitable.

But Is this definition sufficient ? We need much more precise

definition of what a data ware house is .

Page 12: Saranathan Asuri Mphasis

What is a Data Warehouse? A Data warehouse is the Data ( Meta / Fact / Dimension/ Aggregation)

and The Process Managers ( Load / Warehouse /

Query) That make information available , enabling

the user to make informed decisions.

Page 13: Saranathan Asuri Mphasis

Deep Dive

Page 14: Saranathan Asuri Mphasis

What is the Challenge?

Business is never Static. And so is Data Warehouse.

In order to respond to today’s requirement for instant access to corporate information , the data warehouse should be designed to respond to this need in a optimal way.

Business itself probably not aware of what information is required in the future. This requires a fundamentally different

approach than the traditional waterfall method of software development for the Data warehouse.

Page 15: Saranathan Asuri Mphasis

Experience so far…

Most Enterprise Data Warehousing projects tend to have development cycle of between 18 – 24 months from start to finish.

Justification of this investment is substantial. Businesses would prefer a better approach to

justify the investment.

Page 16: Saranathan Asuri Mphasis

What should be done?

Focus on Business Requirements A clear understanding of what is short

term and long term requirement of the data warehouse.

An Architecture design that would evolve.

Identification of quick win that delivers business benefit in the first build.

Page 17: Saranathan Asuri Mphasis

Scalability for Growth

Scalability means ability of the underlying Hardware and Software to support increasing demands over a period of time.

Page 18: Saranathan Asuri Mphasis

Horizontal Scalability

CPU

CPU

CPU

CPU

RAM

RAM

RAM

RAM

DB DB DB DB

CPU

CPU

CPU

CPU

RAM

RAM

RAM

RAM

DBDBDB DB

High Speed Network

Multiple servers are connected thru a network and use the data partitioning feature of the Database to tie the CPUs together.

Page 19: Saranathan Asuri Mphasis

Data Warehouse Environment

Staging Area

Data warehouse(System of Record)

Full History in 3rd Normal Form

No User Access

Summary Area Full HistoryUser Access

Analytical Area

User Access

Source Systems

Data Mart

Page 20: Saranathan Asuri Mphasis

Data Governance

Metadata Mgmt•Architecture•Integration•Control•Delivery

Data Architecture•Entp. DM

• Value Chain

Data Quality• Spec•Analysis

•Measurement•Improvement

Document / Content Mgmt

•Acquisition & Storage•Backup & Recovery

•Content •Retrieval•Retention

DWH / BI•Architecture•Implementatio

n•Training and

Support•Tuning

Data Security•Standards

•Classification•Administration•Authentication

•Auditing

Reference and MDM

•External & Internal Code•Customer Data•Product data

Data Operations•Acquisition•Recovery•Tuning•Purging

Data Development•Analysis

•Data Modeling•DB Design

•Implementation

Strategy

Page 21: Saranathan Asuri Mphasis

Environment

Goals & Objective

s

Technology Activities

Organization & Culture

Roles & Responsibilities

Practices & Techniques Deliverables

Page 22: Saranathan Asuri Mphasis

Architecture Requirements

Must recognize change as a constant Take incremental development approach Existing applications must continue to work Need to allow more data and new types of

data to be added

Page 23: Saranathan Asuri Mphasis

High Level

Remember the different “worlds” On-line transaction processing (OLTP) Business intelligence systems (BIS)

Users are different Data content is different Data structures are different Architecture & methodology must be

different

Page 24: Saranathan Asuri Mphasis

April 22, 2023DW Architecture Best Practices 24

Best Practice #1

Use a Data model that is optimized for Information retrieval dimensional model denormalized hybrid approach

Page 25: Saranathan Asuri Mphasis

April 22, 2023DW Architecture Best Practices 25

Best Practice #2

Carefully design the data acquisition and cleansing processes for your DW Ensure the data is processed efficiently

and accurately Consider acquiring ETL and Data

Cleansing tools Use them well!

Page 26: Saranathan Asuri Mphasis

April 22, 2023DW Architecture Best Practices 26

Best Practice #3

Design a metadata architecture that allows sharing of metadata between components of your DW consider metadata standards such as

OMG’s Common Warehouse Metamodel (CWM)

Page 27: Saranathan Asuri Mphasis

April 22, 2023DW Architecture Best Practices 27

Best Practice #4

Take an approach that consolidates data into ‘a single version of the truth’ Data Warehouse Bus

conformed dimensions & facts OR?

Page 28: Saranathan Asuri Mphasis

April 22, 2023DW Architecture Best Practices 28

Best Practice #5

Consider implementing an ODS only when information retrieval requirements are near the bottom of the data abstraction pyramid and/or when there are multiple operational sources that need to be accessed Must ensure that the data model is

integrated, not just consolidated May consider 3NF data model Avoid at all costs a ‘data dumping ground’

Page 29: Saranathan Asuri Mphasis

Pitfalls to be Avoided

Engagement of Non-BI Manger in a BI delivery Project. Trying to please the client and the user community. Expecting the Service Provider to own the Project completely. Bringing the Solution Architect half way into the project. Allowing the Business Users to drive the Data Model. Not having the right people with right skills in tool selection

process. Expecting the contractor to deliver all that they presented. Over dependency on the Service provider or contractor in

execution. Assuming that the Data quality will be handled somehow. Assuming that the Data warehouse project is over once it is

deployed.

Page 30: Saranathan Asuri Mphasis

Data Warehouse Data Warehouse Architecture Best PracticesArchitecture Best Practices

Thank You