oracle data profiling and quality 11gr1
DESCRIPTION
Oracle Data Profiling and Quality 11gR1 e-SeminarTRANSCRIPT
<I t Pi t H ><Insert Picture Here>
Oracle Data Profiling and Quality 11gR1 Oracle PartnerNetwork EnablementUgo Pollio, Principal Sales Consultant, DIS EMEAFX Ni l P i i l P d t M DISFX Nicolas, Principal Product Manager, DIS
Agendag
• Introduction to Data Quality• Oracle Data Profiling and Quality OverviewOracle Data Profiling and Quality Overview• Data Quality Use Cases
D t ti• Demonstration• Q&A
IntroductionIntroductionData Quality
Managing the Information Sensitive Business
Why do I have so Why are my
Value of Trusted Information
many duplicate copies of data?
Much of it is inaccurate!
Why can’t I get access the data I need for decision
making ?
Why are my applications
referring to last week’s numbers? inaccurate!making ?
Trusted Information
• Accurate
Accessible Information
• Available
Up-to-Date Information
• Fast access • Accurate• Consistent• Quality
• Available• Secure • Reliable
• Fast access• Multiple sources• Actionable
What is Data Quality?y
• Degree of Excellence of the Data• Process of making and keeping data in a state ofProcess of making and keeping data in a state of
completeness, validity, consistency, timeliness and accuracy that makes it appropriate for a specific use.accuracy that makes it appropriate for a specific use.
Example of Data Quality Issuesp y
Matching Records Non Standard formats
Name Address City State Zip Phone Email
Bob Williams 36 Jones Avenue Newton MA 02106 617 555 000 [email protected]
formats
Robert Williams 36 Jones Av. MA 02106 617555000
Burkes, Mike and Ilda 38 Jones av. Nweton MA 02106 617-532-9550 [email protected]
Jason Bourne, 76 East 51st Newton MA 617-536-5480 6175541329Bourne & Cie. 76 East 51 Newton MA 617-536-5480 6175541329
… … … … … … …
Mis-fielded data
Multiple Names
TyposMixed business and contact names
Missing Data
Two Facts about Data QualityyClear view on data
• The Data Quality Challenge is an iceberg• The biggest DQ threats are the ones we do not see.gg Q
Data Profiling lowers the water line and draws a clear view of the quality issues
Known Data
Risk manageableBusiness rules tractableE t ti lIssues
Suspected Data Issues
Expectations clearHigh business user involvement
Unexpected Data Issues Risk unmanageable
Business rules unknowableMi d t tiMissed expectationsLittle business user involvement
Two Facts about Data QualityyData quality decays
• Data value decays• Data is an asset which value decays over timey• Business events can make this worse• Quality is not a one shot process but a constant effort in the y p
enterprise processes.Data Quality needs to be pervasive and continuous.
PervasiveOracle Data Integrator Supplies
Standard, Inline Data Quality and Data Profiling Capabilities with Every ETL and E-LT Job
ContinuousOracle Data Profiling and Oracle Data g
Quality for Data Integrator support Integrated Workflow, Recycling, and Steward GUIs
OverviewOverviewData Profiling & Quality
Oracle Data Quality Productsy
• Oracle Data Integrator• Integrate Data between sources and targets with inline data g g
integrity check• Call Data Quality processes
• Oracle Data Profiling• Investigate quality issues
• Oracle Data Quality for Oracle Data Integrator• Cleanse, Parse, Match & Merge, , g
• Oracle Product Data Quality• Semantic Approach to product data qualitySemantic Approach to product data quality
1. Investigate - Oracle Data Profilingg g
ODP/ODQ Client• Investigate the Data
• Structure
ODP/ODQ Client• Profiling/Investigation• Quality monitoring
• Content• Values, statistics,
frequencies rangesS li & frequencies, ranges• Data Relationships
• Dependencies, keys, Metabase
Sampling &Analysis
p yjoins
• Assess Data Compliance• Report & Alerts• Monitor Quality Over Time
2. Cleanse - Oracle Data Qualityy
ODP/ODQ Client• Proven, scalable DQ
engines
ODP/ODQ Client• Quality Project Design• Rules Tuning
• Rich global content for cleansing, standardization, validationvalidation
• Packaged Quality Rules• Delivered Out-of-the-Box
MetabaseDelivered Out of the Box by Oracle
• For 60+ Countries• Extensible & Customizable
RulesRoute Parse Match Link Merge
Data Quality ServerRoute Parse Match Link Merge
3. Run with Oracle Data Integratorg
• Integration handled by ODI
Oracle Data Integrator• Moves & Transforms Data• Calls Quality Processes
• Advanced quality processing handled by ODQQ
• Pre-built Knowledge Modules• For Metadata Exchange
• Tool for DQ process invocationinvocation
Route Parse Match Link MergeData Quality Server
Route Parse Match Link Merge
What’s New in ODP/ODQ 11gR1g
• Business Rules Library• User-Defined TemplatesUser Defined Templates• Enhanced International Support
• Multi lingual client (user interface for design environment)• Multi-lingual client (user interface for design environment)• Tuning facilities for global data, including double-byte data
• Improved Geographic Support• Improved Geographic Support• E.g.: Expanded Japanese Postal Matcher Options• Latitude/Longitude appends Globally• Latitude/Longitude appends, Globally.
• User Interface Enhancements
Data QualityData QualityUse Cases
Data Quality for Business IntelligenceChange Data into valuable Information
• The Business Issue• BI Reports are not trustable, because of
the state of source data
• Reduce risks• Improve data quality by integrating
DO NOT TRUST THIS
DATA !p q y y g g
cleansing as part of the process• Eliminate data redundancies
Profiling• Investigate
Cleansing
DATA !
• Improve Business Insights• Improved business insight with improved
data quality
Cleansing• Standardize, Enrich, Match
Control• Govern over time q y
• Better profiling of data to eliminate gaps in insight
Data Quality Dashboards in OBIEEyAttribute Analysis, Historical Analysis, DQ Stats
Data Quality FirewallyProfile, Repair, Check, Alert, Report
DatabaseSources
DQ DashboardCOBOL Copybooks
Route Parse Match Link Merge
Files
DiscardedR d
• DWH, improving reliability and quality• New ERP/CRM installation, and legacy data
integration Records
Human Workflow
integration• Master Data Management projects• Data synchronization projects
Data Quality for Migration
ODP + ODQ + ODI
CRM
ODP + ODQ + ODI
Route Parse Match Link Merge
MDM
M&ACOBOL Copybooks
g
Analyze& Design
Build & Cleanse
Test & Validate MDM
ERP
g
Iterations1
DWHDocumentationMetadata
Assumptions IdentifiedFailure(s)
Files 32
1
TomorrowThe Truth
MetadataBusiness Input
Today
Plan (less risk)Managed ResourcesProductive
AgreedScopeTimescaleCost
ContentStructureRelationshipQuality
Demonstration
Questions
For More Information
Quote AttributionTitle, Company
• Visit the Oracle Fusion Middleware 11g
Get Started
• Datasheet:
Resources
gweb site at http://www.oracle.com/goto/fmw11g/index.html
http://www.oracle.com/products/middleware/odi/docs/odiee-datasheet.pdf
• Blog: • Oracle Data Integration on oracle.com
www.oracle.com/goto/odi• Oracle Fusion Middleware on OTN
http://blogs.oracle.com/dataintegration• Technical information available at:
http://www.oracle.com/technology/prodhttp://otn.oracle.com/middleware
• Information on GoldenGate: http://www.oracle.com/goldengate
ucts/oracle-data-integrator/index.html• Data Integration Events
http://www.oracle.com/events