saranathan asuri mphasis

Click here to load reader

Post on 09-Feb-2016

12 views

Category:

Documents

0 download

Embed Size (px)

DESCRIPTION

Saranathan Asuri Mphasis

TRANSCRIPT

  • Asuri Saranathan

  • AgendaIntroductionBest Practices Over ViewDeep DiveConclusionQ & A

  • Introduction

  • SpeakerHolds Bachelor degree in Physics and Electrical and Electronics Engineering.Over 26 years of Experience in Power System and Information Technology field.Has built several large scale applications including online CRM for multinationals.Has managed several Data Warehousing and BI projects for Direct Marketing, Manufacturing and Auto finance verticals.Functioned as Solution Architect for Data warehousing and reporting projects.ISO auditorCertified Bullet Proof Manager from CrestComm USA.

  • Best Practices- Overview

  • What are Best Practices?Is it a technology?Is it application of a set of best tools available in the market?Is it a Framework?

  • Best Practices DefinitionA framework of a set of processes or method that exhibits achievement of specific results in a specific manner over a sustained period of time.The framework should have certain characteristics in that they should be repeatable.

  • Do Best Practices Evolve?Yes they do.Because of Innovation Changes in TechnologyChanges in Law or Governance Structure.Expectations, Values , Knowledge or other that makes the practice outdated or inappropriate.

  • Where can it be Applied?Practically in all fields.

  • How do we apply Best Practices to Data Warehousing and Business Intelligence?

  • Data Warehouse - DefinitionIn an elementary form , it is the collection of key information that can be used by the business users to become more profitable.But Is this definition sufficient ?We need much more precise definition of what a data ware house is .

  • What is a Data Warehouse?A Data warehouse is the Data ( Meta / Fact / Dimension/ Aggregation) andThe Process Managers ( Load / Warehouse / Query) That make information available , enabling the user to make informed decisions.

  • Deep Dive

  • What is the Challenge?Business is never Static.And so is Data Warehouse.In order to respond to todays requirement for instant access to corporate information , the data warehouse should be designed to respond to this need in a optimal way.Business itself probably not aware of what information is required in the future.This requires a fundamentally different approach than the traditional waterfall method of software development for the Data warehouse.

  • Experience so farMost Enterprise Data Warehousing projects tend to have development cycle of between 18 24 months from start to finish. Justification of this investment is substantial.Businesses would prefer a better approach to justify the investment.

  • What should be done?Focus on Business Requirements A clear understanding of what is short term and long term requirement of the data warehouse.An Architecture design that would evolve.Identification of quick win that delivers business benefit in the first build.

  • Scalability for GrowthScalability means ability of the underlying Hardware and Software to support increasing demands over a period of time.

  • Horizontal ScalabilityHigh Speed NetworkMultiple servers are connected thru a network and use the data partitioning feature of the Database to tie the CPUs together.

  • Data Warehouse EnvironmentStaging AreaData warehouse(System of Record)Full History in 3rd Normal FormNo User AccessSummary Area Full HistoryUser AccessAnalytical AreaUser AccessSource SystemsData Mart

  • Data GovernanceMetadata MgmtArchitectureIntegrationControlDeliveryData ArchitectureEntp. DM Value Chain

    Data Quality SpecAnalysisMeasurementImprovementDocument / Content MgmtAcquisition & StorageBackup & RecoveryContent RetrievalRetentionDWH / BIArchitectureImplementationTraining and SupportTuningData SecurityStandardsClassificationAdministrationAuthenticationAuditingReference and MDMExternal & Internal CodeCustomer DataProduct dataData OperationsAcquisitionRecoveryTuningPurgingData DevelopmentAnalysisData ModelingDB DesignImplementationStrategy

  • EnvironmentGoals & ObjectivesTechnologyActivitiesOrganization & CultureRoles & ResponsibilitiesPractices & TechniquesDeliverables

  • Architecture RequirementsMust recognize change as a constantTake incremental development approachExisting applications must continue to workNeed to allow more data and new types of data to be added

  • High Level Remember the different worldsOn-line transaction processing (OLTP)Business intelligence systems (BIS)Users are differentData content is differentData structures are differentArchitecture & methodology must be different

  • *DW Architecture Best Practices*Best Practice #1Use a Data model that is optimized for Information retrievaldimensional modeldenormalizedhybrid approach

    DW Architecture Best Practices

  • *DW Architecture Best Practices*Best Practice #2Carefully design the data acquisition and cleansing processes for your DWEnsure the data is processed efficiently and accuratelyConsider acquiring ETL and Data Cleansing toolsUse them well!

    DW Architecture Best Practices

  • *DW Architecture Best Practices*Best Practice #3Design a metadata architecture that allows sharing of metadata between components of your DWconsider metadata standards such as OMGs Common Warehouse Metamodel (CWM)

    DW Architecture Best Practices

  • *DW Architecture Best Practices*Best Practice #4Take an approach that consolidates data into a single version of the truthData Warehouse Busconformed dimensions & factsOR?

    DW Architecture Best Practices

  • *DW Architecture Best Practices*Best Practice #5Consider implementing an ODS only when information retrieval requirements are near the bottom of the data abstraction pyramid and/or when there are multiple operational sources that need to be accessedMust ensure that the data model is integrated, not just consolidatedMay consider 3NF data modelAvoid at all costs a data dumping ground

    DW Architecture Best Practices

  • Pitfalls to be AvoidedEngagement of Non-BI Manger in a BI delivery Project.Trying to please the client and the user community.Expecting the Service Provider to own the Project completely.Bringing the Solution Architect half way into the project.Allowing the Business Users to drive the Data Model.Not having the right people with right skills in tool selection process.Expecting the contractor to deliver all that they presented.Over dependency on the Service provider or contractor in execution.Assuming that the Data quality will be handled somehow.Assuming that the Data warehouse project is over once it is deployed.

  • Data Warehouse Architecture Best PracticesThank You

    cohesion instituteSeptember 13-14, 2005DW Analysis & Design*