1 copyright © 2013, oracle and/or its affiliates. all rights reserved. · 6 copyright © 2013,...
TRANSCRIPT
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 1
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 2
Oracle Enterprise Data Quality Overview and Roadmap
Martin Boyd – Senior Director, Product Strategy
Mike Matthews – Director, Product Management
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 3
The following is intended to outline our general product direction.
It is intended for information purposes only, and may not be
incorporated into any contract. It is not a commitment to deliver
any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and
timing of any features or functionality described for Oracle’s
products remains at the sole discretion of Oracle.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 4
Program Agenda
Why Care About Data Quality and Governance?
Oracle Enterprise Data Quality
Roadmap and Demonstration
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 5
“Ultimately, poor data quality is like dirt on the windshield. You may be able to drive for a long time with slowly degrading vision, but at some point, you either have to stop and clear the windshield or risk everything.”
Ken Orr, The Cutter Consortium
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 6
Companies
Individuals
Data Changes in the Real-World
Source: D&B, US Census Bureau, US Department of Health and Human Services, Administrative Office
of the US Courts, Bureau of Labor Statistics, Gartner, A.T Kearney, GMA Invoice Accuracy Study
• 5,769 individuals in the US will
change jobs
• 2,748 individuals will change
address
• 515 individuals will get married
• 263 individuals will get divorced
• 186 individuals will declare a
personal bankruptcy
Master data changes at a rate of 2% per month
Products
• On average 20% duplicates in
product data
• 90% product introductions fail
• Retailers lose $40B or 3.5% of total
sales each year due to item master
inaccuracy
• 60% of all invoices will have an
error
• Companies with global data Sync
will realize 30% lower IT costs
In one hour… In one hour… In one year…
6
240 businesses will change
addresses
150 business telephone numbers will
change or be disconnected
112 directorship (CEO, CFO, etc.)
changes will occur
20 corporations will fail
12 new businesses will open their
doors
4 companies will change their name
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 7
Business Impact of Data Quality
With Bad Data With Good Data
• Reduced ROI
• Increased project risk, time and cost
• Expensive downstream consequences –
wrong shipment, wrong invoices,
incorrect parts…
• Increased ROI on existing systems
• Increased agility
• Increased efficiency
• Increased customer satisfaction
• Increased scalability
“Only 30% of BI/DW
implementations fully succeed.
The top two reasons for failure?
Budget constraints and data
quality.”
“Data integration and data quality are
fundamental prerequisites for the
successful implementation of enterprise
applications, such as CRM, SCM, and
ERP.” ”
“#1 reason CRM projects fail:
Data Quality”
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 8
Typical Customer/Party Data Issues
Variation or Error
Example Variation or
Error Example
Sequence errors • Mark Douglas or Douglas Mark Transcription
mistakes • Hannah, Hamah
Involuntary corrections
• Browne – Brown Missing or extra
tokens • George W Smith, George Smith, Smith
Concatenated names
• Mary Anne, Maryanne Foreign sourced
data
• Khader AL Ghamdi, Khadir A.
AlGamdey
Nicknames and aliases
• Chris – Christine, Christopher, Tina Unpredictable
use of initials • John Alan Smith, J A Smith
Noise • Full stops, dashes, slashes, titles,
apostrophes Transposed
characters • Johnson, Jhonson
Abbreviations
• Wlm/William, Mfg/Manufacturing Localization • Stanislav Milosovich – Stan Milo
Truncations • Credit Suisse First Bost Inaccurate dates • 12/10/1915, 21/10/1951, 10121951,
00001951
Prefix/suffix errors
• MacDonald/McDonald/Donald Transliteration
differences • Gang, Kang, Kwang
Spelling & typing
errors • P0rter, Beht Phonetic errors • Graeme – Graham
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 9
Typical Product/Item Data Issues
10hp motor 115V Yoke mount
mtr, ac(115) 10 horsepower 115volts
MOT-10,115V, 48YZ,YOKE
This 10hp yoke mounted motor is rated for
115V with a 5 year warranty
10 Caballos, Motor, 115 Voltios
TEAO HP = 10.0 1725RPM 115V 48YZ YOKE MTR
Motor, TEAO, 1725 RPM, 48YZ, 15 Voltios,
Montaje de Yugo, hp = 10
Item Motor
Classification 26101600
Power 10 horsepower
Voltage 115
Mounting Yoke
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 10
Putting your Data to Work Common Data Quality Use Cases
System Consolidation/Migration
• Enforce new system standards on
legacy data
Compliance
• Drive consistent data and processes
to meet regulatory requirements
(watchlist screening, anti-money
laundering, tax compliance, etc.)
Application Enablement
• Clean-up and govern application
data (CRM, HR, PLM, Retail
search, etc.)
Business Intelligence Enablement
• Enforce BI standards on disparate
data
MDM Enablement
• Verify, standardize, match and
and merge data from disparate
sources
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 11
• How do you know?
• What is the business impact?
• What should you do about it?
Data Quality – Is Your Data “Fit for Purpose”?
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 12
Health Check – Is Your Data “Fit for Purpose”?
Understand current data ‘fitness for purpose’
Estimate DQ impacts & ROI
Identify critical issues & quick wins Understand
Improve
Protect
Govern Your
Data
Your Experts
Current
issues,
gaps,
errors
Business &
data
standards
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 13
Improve Data, Improve App Performance
Improve ROI and performance of existing applications
Engage users and executives
Bring data to a known, baseline quality – ready to roll-out new
applications and initiatives
Understand
Improve
Protect
Govern
Metrics,
KPIs
Fit for
purpose
data
Parse/
extract
Stand-
ardize
Match/
merge
Verify
Enrich
‘Gold’
data
Apply data standards
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 14
‘DQ Firewall’ – Continuous Protection for Information Assets
Continuous, consistent enforcement of standards
High quality data drives ROI
No more DQ projects!
Understand
Improve
Protect
Govern
Hub
Apply data standards/validate
External
sources/
feeds
Data Integration/ETL Non-DQ/MDM-
aware Apps
DQ/MDM-
aware Apps Web
service
call
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 15
DQ Governance – Continuous Process Improvement
Monitor ongoing effectiveness
Track and resolve issues
Improve overall effectiveness
Understand
Improve
Protect
Govern
Target
system DQ
metrics
‘Gold’
data
Apply data standards
Source
system DQ
metrics
DQ
process
metrics
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 16
Program Agenda
Why Care About Data Quality and Governance?
Oracle Enterprise Data Quality
Roadmap and Demonstration
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 17
Modernization MDM SOA Big Data
Oracle Data Integration Complete Offering for Enterprise Data Integration
Complete and best-of-breed
approach for enterprise data
integration
Maximum performance with
lower TCO, ease of use and
reliability
Certified for leading
technologies to deliver fast
time to value
Oracle Data Integrator
Oracle GoldenGate
Oracle Enterprise Data Quality
Oracle Data Service Integrator
OLTP
Applications
Legacy
Unstructured
Synchronization Custom BI
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 18
Enterprise Data Quality
• Process metrics
• Quality metrics
• Case Management
• Remediation
• Party (individuals,
households) match
• Entity match
• Semantic (category)
match
• Statistical match
• Match review
• Merge/survivorship
• Global parse
• Category parse
• Extract
• Transform
• Address verification &
geocoding
• Substitute
• Enrich
• Classify
• Statistics
• Patterns
• Phrases
• Duplicates
• Completeness
• Max/min values
Profile
Standardize
Match
Govern
Quickly understand data content
Drive conformance to standards
Identify & merge duplicates
Monitor effectiveness & resolve problems
Co
mm
on
Acce
ss/U
I
Enterprise DQ Platform
Enterprise DQ Cloud Services
• Packaged cloud services for cloud applications Enterprise DQ
Matching Cloud
Service
Enterprise DQ
Address Verification
Cloud Service
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 19
Enterprise Data Quality
Broadest DQ offering • Best of breed capabilities for both Party Data and Product
Data
• Profiling, standardization, matching, case management,
governance
Most usable DQ offering • Completely integrated offering – designed to work together
• Designed for business and technical users
• Transparent operation and results – no black boxes
Pervasive operation for enterprise quality governance • Within legacy systems and MDM Hubs
• As part of migration/system load
• On data entry/capture
• As part of data movement/transfer
Profile
Standardize
Match
Govern
Quickly understand data content
Drive conformance to standards
Identify & merge duplicates
Monitor effectiveness & resolve problems
Co
mm
on
Acce
ss/U
I
Enterprise DQ Platform
Enterprise DQ Cloud Services Enterprise DQ
Matching Cloud
Service
Enterprise DQ
Address Verification
Cloud Service
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 20
EDQ Web Services Enforce common DQ standards across the enterprise
Common
Services
Applications App 1 App 2 App 3
Library of
enterprise
standard DQ
services
Any EDQ process may
be called as a real-time
web service
Call any process from
any application to
1. Enforce common
standards
2. Minimize
architectural
changes
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 21
Case Management for Governance
Usage • Cases/alerts are assigned a work queues and a priority
• Data specialists sign in and review/resolve issues
• Management reports allow monitoring of work queues and productivity
• Helpful for
o One-time cleanse/migration
o Ongoing governance program
Features • Hierarchical Case/alert functionality
• Configurable Workflows
• Automatic prioritization of cases/alerts
• Timers
• Email Notification Support
• Comprehensive audit trail
• Immediate ad-hoc reporting
Review and resolve exceptions from the DQ process
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 22
Data Prep for System Migration/Implementation
EDQ Process
Apps and hubs
Governance and Case Management to ‘Perfect’ Data
• DQ Insight (Dashboard)
• Reporting
• Trend Analysis
• Case Management
• Workflow
• Remediation
Legacy Data ‘Fit for Purpose’ Data
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 23
Program Agenda
Why Care About Data Quality and Governance?
Oracle Enterprise Data Quality
Roadmap and Demonstration
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 24
EDQ Investment Areas
Integrated DQ &
Governance
Integrated best-in-class Customer and
Product DQ
Expand Governance to include
operational confidence reporting
Integration Across Oracle
Deeper Siebel Integration
Out-of-the-box DQ for Fusion Apps
Integration with ODI
Endeca, ATG, EBS…
Cloud/SaaS
SaaS deployment for Fusion Apps
Full clustering and elastic provisioning
support
Cloud DQ Services
Global Rules & UI
Global Identity Resolution
DQ Rules and Reference Data for
major locales
Additional UI Localizations
Advanced Techniques
Statistical parsing & classification
Statistical outlier detection
Entity identification & extraction for Big
Data
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 25
EDQ in the Cloud
Cloud Data Services powered by EDQ
• Providing data enhancement services in the Oracle Cloud
• Uses EDQ as the matching engine and to ensure reference
data quality
EDQ in Fusion Apps
• EDQ to be deployed and used by Fusion Apps
• Leveraging Oracle DB and FMW cloud support
EDQ in Managed Cloud
• Growing number of customers already choosing to run full service EDQ in the Oracle Managed Cloud
EDQ powering Partner Cloud Offerings
• Kaygen partnering with Oracle to deliver Data Governance in managed cloud with EDQ
• Several others following suit
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 26
EDQ for Fusion Applications
Fusion Applications
Integration
– EDQ deployed in Fusion
Apps as the attached DQ
engine
– Advanced Search, Duplicate
Prevention, Master Data
Matching
– Address Verification and
Cleaning for all countries Profile
Standardize
Match
Govern
Quickly understand data content
Drive conformance to standards
Identify & merge duplicates
Monitor effectiveness & resolve problems
Co
mm
on
Acce
ss/U
I
Enterprise DQ Platform
Enterprise DQ Cloud Services Enterprise DQ
Matching Cloud
Service
Enterprise DQ
Address Verification
Cloud Service
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 27
EDQ 11 - Major New Features
Case Management Expansion
– Instant reports on high volume data
– Aggregated reports (e.g. activity by period, priority, etc.)
– Improved case search and filter
– Expanded workflow options
Reference Published Processors
– Enables development of ‘locked’ IP to extend EDQ
– Full reuse and upgrade of processors across processes/projects
UI Localization to 9 Languages
– Chinese, Japanese, Korean, Brazilian Portuguese, French, Italian, German,
Spanish, English
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 28
EDQ 11 - Improving Productivity
New Job Manager
– User-defined job layouts and canvas notes
– ‘Blocking’ triggers allow jobs to be called within jobs with
execution control
– Additional externalization options
New Process Canvas
– Improved canvas usability and multi-language support
Browser-based Web Service Tester
– Faster testing of EDQ Web Services
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 29
EDQ 11 – Other Changes
Oracle Universal Installer
– Automated installation process for all platforms
Fusion Middleware Integration
– Enables use of WebLogic OPSS for security and authentication
– Uses FMW Audit Control to capture key configuration changes
Automated Results Purge capability
Support for Subversion 1.7
Array support in Data Interfaces
Multi-attribute data type converters
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 30
EDQP 11 - Major Features New Integrations
– Connector for Endeca Guided Navigation
– Integrated with Agile PLM 9.3.2
Statistical Matching Module
(StatSim)
– Quick Rules Free Configuration
– Match or classify verbose semi-structured
data
– Integrated with Governance Studio
Remediation Capabilities
– Provides List of Values for Data
Enrichment
– Integrated with AutoLearn Workflow
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 31
EDQP Drives Endeca Navigation
Integrated Data Quality:
– Populate – Identify, extract and standardize product
dimensions & properties
– Integrate – Automatically create required dimensions
within Endeca (avoid manual dimension setup)
Endeca
Engine
EDQP
Client
Browser Data
Source Data
Source
Data
Source
Improved data improves user experience
Standardize data structure
Standardize data values
Integrated directly into Endeca pipeline
Endeca
Load
Data Preparation
EDQP ‘pushes’ required metadata
into Endeca to create required
navigation dimensions
PIM or any other
data source
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 32
EDQ 12c
Data Quality Governance II
Integrated Semantic Data Engine (EDQ-P)
Full WebLogic Server Clustering support
– Shared config for multiple EDQ servers
– Session balancing and failover
– Active-Active Case Management
Oracle Access Manager integration
Hadoop Connectivity
Fully automatable Reference Data Generation
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 33
Data Governance with EDQ
Single DQ environment
DQ Engine
Data sources
Real-time checks
Apps and hubs
Enabling People and Process with Technology
• DQ Insight (Dashboard)
• Reporting
• Trend Analysis
• Case Management
• Workflow
• Remediation
Current capabilities to
be enhanced and
combined into a new
cloud-enabled DQ
Governance UI
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 34
EDQ Application Integration
• Fusion Applications – Deep integration in progress; planned for Fusion R9 release
• Siebel CRM and UCM– Deep integration in place using services architecture; more stable,
performant, functional and scalable than 3rd Party or OEM integrations
• EBS – Template connectors available for common integrations (customer/party, etc.)
• Salesforce.com – Template connectors available for batch cleansing
• Oracle Product Hub; Fusion Product Hub – Deep integration for batch and real-time load
• ENDECA (Oracle Commerce) – Data cleansing and metadata sync to streamline managing complex
product data schemas for eCommerce
• Agile PLM – Template connectors for batch and real-time validation, BOM validation and BOM sync
Enabling Applications with Data Quality Services
• Application owners are painfully aware of the impact & costs of poor data
• EDQ is investing heavily in providing out-the-box Application DQ solutions
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 35
Join the Data Integration and MDM Community
Twitter Facebook Blog LinkedIn YouTube
blogs.oracle.com/dataintegration
blogs.oracle.com/mdm
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 36
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 37