data quality for successful data...

1

Informatica Confidential - 2008

Ivan ChongEVP and General ManagerData Quality Business UnitInformatica Corporation

Data Quality for Successful Data Governance

2Informatica Confidential - 2008

The MIT 2008 Information Quality Industry Symposium

Agenda

• Introduction

• A Pragmatic Approach

• Common Pitfalls and Misconceptions

• Examples / Scenarios & Case Studies

• Q&A

MIT Information Quality Industry Symposium, July 16-17, 2008

321



Introduction:Data Quality Evolution

ContactEfficiencyContact

EfficiencyRelationshipIdentificationRelationshipIdentification

Customized Data

Quality

Customized Data

Quality

Data QualityAnalysis

Data QualityAnalysis

Enterprise DQ Suite

Parsing, standardization, cleansing and validation of customercontact-related data elements, such as names, addresses and telephone numbers

Parsing, standardization, cleansing and validation of customercontact-related data elements, such as names, addresses and telephone numbers

Discerning relationships between records (for example,householding, de-duplicating or linking)

Discerning relationships between records (for example,householding, de-duplicating or linking)

Data quality operations for a range of data subject areas, including product data and financial data

Data quality operations for a range of data subject areas, including product data and financial data

Data profiling, measurement and quantification of data quality impact, and ongoing auditing and monitoring of data quality

Data profiling, measurement and quantification of data quality impact, and ongoing auditing and monitoring of data qualitySCM PIM / MDM

SOX

Six Sigma

Compliance

Solutions



Bad Credit DecisionsBad Credit DecisionsBad Credit Decisions

Trusted Information

Trusted Trusted InformationInformation

AML Detection

AML AML DetectionDetection

New Products To Market

New Products New Products To MarketTo Market

Customer Service

Customer Customer ServiceService

Time to introducenew trading partner

Penalties from Regulators

Organizationalmistrust

Difficulty in decision making

Effective Marketing

Risk exposure

Lost customer loyalty

Introduction:Impact of Poor Data Processes & Data Quality

Poor Data Quality or inconsistent data causes defects in the value chain and is a momentum killer for any Data Alignment, Synchronization and Collaboration project, Master Data Management, CDI, Application Implementation etc …


322



Governing corporate data may be slightly easier than governing nations, but companies can benefit from adopting similar

strategies, according to a Gartner analyst.

Executive level. sponsorship, strategic direction, funding, advocacy and oversight

Judicial level. planning activities and to enforce governance activities or corporate policies. Mediating disagreements

Legislative level. chaired by a senior business leader designated by the executive team and may include business and technology leaders from Finance, IT, Data Management and Operations

Administrative level. Implement data governance on a day-to-day basis; responsible for developing data models and corporate datavocabularies, implementing master data management best practices, organizing content

Introduction:One Perspective to Governance, top down

Top Down – often initiated by Exec initiative – Compliance / Audit / BPR / Six Sigma


The MIT 2008 Information Quality Industry SymposiumIntroduction:Data Quality Perspective to Governance

Lack of a comprehensive Data Quality Management Strategy and poor Data Quality is causing Master Data Management (MDM) initiatives to fail or be significantly delayed.

key elements of effective Data Quality Management:• capable data processes

• ownership and accountability

• standards and metrics

• measurement and control

key stages in any migration or MDM implementation project• discovery and assessment

• planning and preparation

• cleansing, alignment, enrichment and enhancement

• operational impact and support in preparation for, during and after implementation

• Transition management• post implementation measurement and control

Bottom Up – often initiated by project – DQ / Migration / App Implement / Six Sigma

Operational

Risk


323



Agenda

• Introduction




• Q&A



ALIGN

Data Quality Management –A Good Starting Point

• Understand your source data and where it resides• Identify key data risks and issues• Define Scorecard / Matrix for key data attributes

• Define business rules to enhance data • Use external content to validate, enhance data• Initiate data cleansing projects

• Enforce guidelines to help align and standardize data• Use internal, external (Cust) and Industry standards• Where to hold your Golden Copy of Master Data?

• Scorecard data quality KPIs • Empower business users • Data Quality firewall

ANALYZE

ALIGN

CLEANSE

SUSTAIN

DQ

M -R

OA

DM

AP


324



Agenda

• Introduction




• Q&A



Background:Focus on value early

Poor Data Quality or inconsistent data causes defects in the value chain and is a momentum killer for any Data Alignment, Synchronization, MDM and Collaboration project

The impacts of poor quality data are not always highly visible. Using Supply Chain as an example here, many impacts are hidden ‘below the waterline’. The

visible impacts represent the ‘tip of the iceberg’

When considering an inventory reduction

initiative, 13% of savings were based on

improving data quality

Decreased by €1.2m annually in XYZ by improving customer delivery time data


325



Information Governance FrameworkBe clear of what is in scope for ‘DATA GOVERNANCE’

Technology Governance• Data Security

• Anti-virus• Disaster Recovery

Information Governance• Data Protection• Licensed 3rd PTY Data• Legislation

Data Governance• Business Processes• Data Quality• Roles and Response’• Business context



MDM – Generic concerns

MDM Applications / Solutions

•Decouple Master and Reference Data capture from Legacy

•All Master Data captured in one application process across the enterprise

•Operational Data Stores and Data Warehouse Risks

Implementation Risks

• Registry / Repository / Hub

• Focus on quick wins and value

• Operational Risk, Internal & External


326



The Challenge Data Integration Complexity



Early Analysis is critical – Obsolescence / DuplicatesThe Challenge


327



One Customers Experience - Large US Bank



Agenda

• Introduction




• Q&A


328

17


Metrics and Measurement



Six Types of Data Quality Problems

Completeness What data is missing or unusable?

Conformity What data is stored in a non-standard format?

Consistency What data values give conflicting information?

Accuracy What data is incorrect or out of date?

Duplicates What data records or attributes are repeated?

Integrity What data is missing or not referenced?


329



Data Quality Certification Scorecard Attributes Within Object with context

Products

Customer

External Data

Credit Rating

Completeness

Conformity

Consistency

Bad Record Reports

Issue Reports

Scorecard



Data Quality Certification - Scorecard per Object

Customer Service

AML

Reconciliation

Credit Risk

Suppression List

Customer Data Scorecard


330



Example – CPG - What is Data Quality?

Completeness

Standards Based

Consistency

Accuracy

Time Stamped

Required values electronically recorded

Data conforms to industry standards

Data values aligned across systems

Data values are right, at the right time

Validity timeframe of data is clear



Customer Example: AML


331



Scorecard: Using ‘Business’ terms



Business / Project Mgnt.

ScorecardProgress reports

ScorecardProgress reports

Business SMEs

Bad Record ReportsIssue Reports

Bad Record ReportsIssue Reports

Global CPG Organization: Quality Check in ETL

Data Quality SolutionQuality Check

Data Quality SolutionQuality Check

ETL ProcessETL Process

Data Experts / Owners take appropriate actions

Data Experts / Owners analyze

reportsDefine business rules for:• Correctness• Completeness• Standardization• Duplicates

Implement business rules

Legacy SAP

RulesRepository

Reports

1

2

3

5

6

4


332



Global CPG Organization: Quality Check in ETL

ETL ProcessETL Process

Define business rules for:

• Correctness• Completeness• Standardization• Duplicates

Legacy SAP

2

Data Experts / Owners take

appropriate actions

Data Experts / Owners analyze reports

Extract Data

ImplementBusiness

Rules

3'

3

Execute IDQ Plans

4

5Extract Data 3'

3

Execute IDQ Plans

1

Data Experts / Owners analyze reports 4

Data Experts / Owners take

appropriate actions

5



DQ must deliver value


333



DQ underpins Data Governance

A.N.Other CPG Org.

A.N.Other



Agenda

• Introduction




• Q&A


334

29


Questions ?

Thank You


335

data quality for successful data...

Documents