oracle data profiling and quality 11gr1

<I t Pi t H ><Insert Picture Here>

Oracle Data Profiling and Quality 11gR1 Oracle PartnerNetwork EnablementUgo Pollio, Principal Sales Consultant, DIS EMEAFX Ni l P i i l P d t M DISFX Nicolas, Principal Product Manager, DIS

Agendag

• Introduction to Data Quality• Oracle Data Profiling and Quality OverviewOracle Data Profiling and Quality Overview• Data Quality Use Cases

D t ti• Demonstration• Q&A

IntroductionIntroductionData Quality

Managing the Information Sensitive Business

Why do I have so Why are my

Value of Trusted Information

many duplicate copies of data?

Much of it is inaccurate!

Why can’t I get access the data I need for decision

making ?

Why are my applications

referring to last week’s numbers? inaccurate!making ?

Trusted Information

• Accurate

Accessible Information

• Available

Up-to-Date Information

• Fast access • Accurate• Consistent• Quality

• Available• Secure • Reliable

• Fast access• Multiple sources• Actionable

What is Data Quality?y

• Degree of Excellence of the Data• Process of making and keeping data in a state ofProcess of making and keeping data in a state of

completeness, validity, consistency, timeliness and accuracy that makes it appropriate for a specific use.accuracy that makes it appropriate for a specific use.

Example of Data Quality Issuesp y

Matching Records Non Standard formats

Name Address City State Zip Phone Email

Bob Williams 36 Jones Avenue Newton MA 02106 617 555 000 bob.williams@yahoo.com

formats

Robert Williams 36 Jones Av. MA 02106 617555000

Burkes, Mike and Ilda 38 Jones av. Nweton MA 02106 617-532-9550 mburkes@gmail.com

Jason Bourne, 76 East 51st Newton MA 617-536-5480 6175541329Bourne & Cie. 76 East 51 Newton MA 617-536-5480 6175541329

… … … … … … …

Mis-fielded data

Multiple Names

TyposMixed business and contact names

Missing Data

Two Facts about Data QualityyClear view on data

• The Data Quality Challenge is an iceberg• The biggest DQ threats are the ones we do not see.gg Q

Data Profiling lowers the water line and draws a clear view of the quality issues

Known Data

Risk manageableBusiness rules tractableE t ti lIssues

Suspected Data Issues

Expectations clearHigh business user involvement

Unexpected Data Issues Risk unmanageable

Business rules unknowableMi d t tiMissed expectationsLittle business user involvement

Two Facts about Data QualityyData quality decays

• Data value decays• Data is an asset which value decays over timey• Business events can make this worse• Quality is not a one shot process but a constant effort in the y p

enterprise processes.Data Quality needs to be pervasive and continuous.

PervasiveOracle Data Integrator Supplies

Standard, Inline Data Quality and Data Profiling Capabilities with Every ETL and E-LT Job

ContinuousOracle Data Profiling and Oracle Data g

Quality for Data Integrator support Integrated Workflow, Recycling, and Steward GUIs

OverviewOverviewData Profiling & Quality

Oracle Data Quality Productsy

• Oracle Data Integrator• Integrate Data between sources and targets with inline data g g

integrity check• Call Data Quality processes

• Oracle Data Profiling• Investigate quality issues

• Oracle Data Quality for Oracle Data Integrator• Cleanse, Parse, Match & Merge, , g

• Oracle Product Data Quality• Semantic Approach to product data qualitySemantic Approach to product data quality

1. Investigate - Oracle Data Profilingg g

ODP/ODQ Client• Investigate the Data

• Structure

ODP/ODQ Client• Profiling/Investigation• Quality monitoring

• Content• Values, statistics,

frequencies rangesS li & frequencies, ranges• Data Relationships

• Dependencies, keys, Metabase

Sampling &Analysis

p yjoins

• Assess Data Compliance• Report & Alerts• Monitor Quality Over Time

2. Cleanse - Oracle Data Qualityy

ODP/ODQ Client• Proven, scalable DQ

engines

ODP/ODQ Client• Quality Project Design• Rules Tuning

• Rich global content for cleansing, standardization, validationvalidation

• Packaged Quality Rules• Delivered Out-of-the-Box

MetabaseDelivered Out of the Box by Oracle

• For 60+ Countries• Extensible & Customizable

RulesRoute Parse Match Link Merge

Data Quality ServerRoute Parse Match Link Merge

3. Run with Oracle Data Integratorg

• Integration handled by ODI

Oracle Data Integrator• Moves & Transforms Data• Calls Quality Processes

• Advanced quality processing handled by ODQQ

• Pre-built Knowledge Modules• For Metadata Exchange

• Tool for DQ process invocationinvocation

Route Parse Match Link MergeData Quality Server

Route Parse Match Link Merge

What’s New in ODP/ODQ 11gR1g

• Business Rules Library• User-Defined TemplatesUser Defined Templates• Enhanced International Support

• Multi lingual client (user interface for design environment)• Multi-lingual client (user interface for design environment)• Tuning facilities for global data, including double-byte data

• Improved Geographic Support• Improved Geographic Support• E.g.: Expanded Japanese Postal Matcher Options• Latitude/Longitude appends Globally• Latitude/Longitude appends, Globally.

• User Interface Enhancements

Data QualityData QualityUse Cases

Data Quality for Business IntelligenceChange Data into valuable Information

• The Business Issue• BI Reports are not trustable, because of

the state of source data

• Reduce risks• Improve data quality by integrating

DO NOT TRUST THIS

DATA !p q y y g g

cleansing as part of the process• Eliminate data redundancies

Profiling• Investigate

Cleansing

DATA !

• Improve Business Insights• Improved business insight with improved

data quality

Cleansing• Standardize, Enrich, Match

Control• Govern over time q y

• Better profiling of data to eliminate gaps in insight

Data Quality Dashboards in OBIEEyAttribute Analysis, Historical Analysis, DQ Stats

Data Quality FirewallyProfile, Repair, Check, Alert, Report

DatabaseSources

DQ DashboardCOBOL Copybooks

Route Parse Match Link Merge

Files

DiscardedR d

• DWH, improving reliability and quality• New ERP/CRM installation, and legacy data

integration Records

Human Workflow

integration• Master Data Management projects• Data synchronization projects

Data Quality for Migration

ODP + ODQ + ODI

CRM

ODP + ODQ + ODI

Route Parse Match Link Merge

MDM

M&ACOBOL Copybooks

g

Analyze& Design

Build & Cleanse

Test & Validate MDM

ERP

g

Iterations1

DWHDocumentationMetadata

Assumptions IdentifiedFailure(s)

Files 32

1

TomorrowThe Truth

MetadataBusiness Input

Today

Plan (less risk)Managed ResourcesProductive

AgreedScopeTimescaleCost

ContentStructureRelationshipQuality

Demonstration

Questions

For More Information

Quote AttributionTitle, Company

• Visit the Oracle Fusion Middleware 11g

Get Started

• Datasheet:

Resources

gweb site at http://www.oracle.com/goto/fmw11g/index.html

http://www.oracle.com/products/middleware/odi/docs/odiee-datasheet.pdf

• Blog: • Oracle Data Integration on oracle.com

www.oracle.com/goto/odi• Oracle Fusion Middleware on OTN

http://blogs.oracle.com/dataintegration• Technical information available at:

http://www.oracle.com/technology/prodhttp://otn.oracle.com/middleware

• Information on GoldenGate: http://www.oracle.com/goldengate

ucts/oracle-data-integrator/index.html• Data Integration Events

http://www.oracle.com/events