the ep technical data quality metrics...
TRANSCRIPT
Presentation Title (acronym) ; Division – Name/OPU/HCU/BU (acronym) ; Name of Presenter ©Petroliam Nasional Berhad (PETRONAS) 2014 1
Presentation Title: DQ Metrics Architecture
Division – PETRONAS
Name of Presenter : Philip Lesslar
© 2014 PETROLIAM NASIONAL BERHAD (PETRONAS) All rights reserved. No part of this document may be reproduced, stored in a retrieval system or transmitted in any form or by any means (electronic, mechanical, photocopying, recording or otherwise) without the permission of the copyright owner.
The EP Technical Data Quality Metrics Initiative
Philip Lesslar, Technical Assurance, Technical Data Digital Energy Conference 13th October 2014 Impiana Hotel, Kuala Lumpur
Architectural Considerations for Global Governance
Presentation Title (acronym) ; Division – Name/OPU/HCU/BU (acronym) ; Name of Presenter ©Petroliam Nasional Berhad (PETRONAS) 2014 2
Presentation Title: DQ Metrics Architecture
Division – PETRONAS
Name of Presenter : Philip Lesslar
Presentation Title: DQ Metrics Architecture
Division – PETRONAS
Name of Presenter : Philip Lesslar
• Objectives • Requirements for global governance • Key supporting initiatives • Architecture elements • Global governance readiness • Dashboard designs • Concluding remarks
Contents
Presentation Title (acronym) ; Division – Name/OPU/HCU/BU (acronym) ; Name of Presenter ©Petroliam Nasional Berhad (PETRONAS) 2014 3
Presentation Title: DQ Metrics Architecture
Division – PETRONAS
Name of Presenter : Philip Lesslar
Presentation Title: DQ Metrics Architecture
Division – PETRONAS
Name of Presenter : Philip Lesslar
Contents
• Objectives • Requirements for global governance • Key supporting initiatives • Architecture elements • Global governance readiness • Dashboard designs • Concluding remarks
Presentation Title (acronym) ; Division – Name/OPU/HCU/BU (acronym) ; Name of Presenter ©Petroliam Nasional Berhad (PETRONAS) 2014 4
Presentation Title: DQ Metrics Architecture
Division – PETRONAS
Name of Presenter : Philip Lesslar
Presentation Title: DQ Metrics Architecture
Division – PETRONAS
Name of Presenter : Philip Lesslar
• Show the relevance of architecture in addressing business problems
• Explain PETRONAS’s Data Quality Metrics initiative within this context
Objectives
Presentation Title (acronym) ; Division – Name/OPU/HCU/BU (acronym) ; Name of Presenter ©Petroliam Nasional Berhad (PETRONAS) 2014 5
Presentation Title: DQ Metrics Architecture
Division – PETRONAS
Name of Presenter : Philip Lesslar
Presentation Title: DQ Metrics Architecture
Division – PETRONAS
Name of Presenter : Philip Lesslar
Data
collation
Project
set up
Workflow and
Standardization
Project
Database
Application Source / feeder
databases
Master
(Reference) Data
Data Quality
Metrics
Corporate
Database
Publishing
Re-use
Previous industry studies : Geologists spend up to 60% of their time looking for & quality checking data for their projects
Re-use
Value Loop
Value Loop
Corporate Data
Cost management
Process improvement
Optimization
Reduce the 39 days to 5 mins
for the 4 data types.
Standards & workflows
Bring to acceptable quality
Capture work done for re-use
‘One version of truth’ datasets
Re-use many times
1
1
1
2
2 2
2
3
3
3
Approved
& final
Feeding new projects
unique master source
of quality checked data
The project space is where
the big decisions are made
Reduction of time for data
collation and project setup
Step 1
Step 2
Step 3
The Role of Data Management in Value Creation
Recent PETRONAS study : Geologists spend 39 days looking for & quality checking basic data (well header, checkshots,
deviation and well logs) for their projects. This is an estimated 20% of their time.
Internal
Presentation Title (acronym) ; Division – Name/OPU/HCU/BU (acronym) ; Name of Presenter ©Petroliam Nasional Berhad (PETRONAS) 2014 6
Presentation Title: DQ Metrics Architecture
Division – PETRONAS
Name of Presenter : Philip Lesslar
Presentation Title: DQ Metrics Architecture
Division – PETRONAS
Name of Presenter : Philip Lesslar
Contents
• Objectives • Requirements for global governance • Key supporting initiatives • Architecture elements • Global governance readiness • Dashboard designs • Concluding remarks
Presentation Title (acronym) ; Division – Name/OPU/HCU/BU (acronym) ; Name of Presenter ©Petroliam Nasional Berhad (PETRONAS) 2014 7
Presentation Title: DQ Metrics Architecture
Division – PETRONAS
Name of Presenter : Philip Lesslar
Presentation Title: DQ Metrics Architecture
Division – PETRONAS
Name of Presenter : Philip Lesslar
• Central data quality team
• DQ tool with global reach
• Ownership & accountability framework
• Standardized application and database portfolio
Minimum Requirements for global governance
Presentation Title (acronym) ; Division – Name/OPU/HCU/BU (acronym) ; Name of Presenter ©Petroliam Nasional Berhad (PETRONAS) 2014 8
Presentation Title: DQ Metrics Architecture
Division – PETRONAS
Name of Presenter : Philip Lesslar
Presentation Title: DQ Metrics Architecture
Division – PETRONAS
Name of Presenter : Philip Lesslar
Contents
• Objectives • Requirements for global governance • Key supporting initiatives • Architecture elements • Global governance readiness • Dashboard designs • Concluding remarks
Presentation Title (acronym) ; Division – Name/OPU/HCU/BU (acronym) ; Name of Presenter ©Petroliam Nasional Berhad (PETRONAS) 2014 9
Presentation Title: DQ Metrics Architecture
Division – PETRONAS
Name of Presenter : Philip Lesslar
Presentation Title: DQ Metrics Architecture
Division – PETRONAS
Name of Presenter : Philip Lesslar
• Data standards program • Data type standards, databases, reference data etc
• Data quality metrics initiative
• Business rules for - Data type (completeness, integrity, workflow), projects (structure), databases (footprint compliance), standards (compliance)
• Governance dashboards for transparency
• Technical Data Framework (TDF) rollout
• Guidelines, roles & responsibilities, accountability structure
• Application standardization
• Technical Applications Portfolio Management
Key supporting initiatives
Presentation Title (acronym) ; Division – Name/OPU/HCU/BU (acronym) ; Name of Presenter ©Petroliam Nasional Berhad (PETRONAS) 2014 10
Presentation Title: DQ Metrics Architecture
Division – PETRONAS
Name of Presenter : Philip Lesslar
Presentation Title: DQ Metrics Architecture
Division – PETRONAS
Name of Presenter : Philip Lesslar
Contents
• Objectives • Requirements for global governance • Key supporting initiatives • Architecture elements • Global governance readiness • Dashboard designs • Concluding remarks
Presentation Title (acronym) ; Division – Name/OPU/HCU/BU (acronym) ; Name of Presenter ©Petroliam Nasional Berhad (PETRONAS) 2014 11
Presentation Title: DQ Metrics Architecture
Division – PETRONAS
Name of Presenter : Philip Lesslar
Presentation Title: DQ Metrics Architecture
Division – PETRONAS
Name of Presenter : Philip Lesslar
IQM - Architecture
Query engine in central
runs against central and
remote databases
Query engine writes metrics
back to central database
Web clients from anywhere on
the company network can
configure IQM and see results
IQM Web server and the IQM
database are centralized
Web Client
Batch engine
Database
Web forms
EP Data
Configuration/results DB
Web Client
Metrics Engine
IIS (Internet Information
Services) Web server
Configuration/reporting
Central Remote
User interface
Oracle Oracle Oracle Oracle Oracle Oracle
The query engine can
run once, as a
nominated global user,
or multiple times,
against each remote
site
Diagram courtesy of Exprodat Inc.
Presentation Title (acronym) ; Division – Name/OPU/HCU/BU (acronym) ; Name of Presenter ©Petroliam Nasional Berhad (PETRONAS) 2014 12
Presentation Title: DQ Metrics Architecture
Division – PETRONAS
Name of Presenter : Philip Lesslar
Presentation Title: DQ Metrics Architecture
Division – PETRONAS
Name of Presenter : Philip Lesslar
Data Quality Metrics Aggregation (concept)
Single Query
eg. select well_name from well_master;
PROJECT
DATABASE
TARGET
Single Query
Single Query
Single Query
Single Query
Query Set
PROJECT
DATABASE PROJECT
DATABASE
PROJECT
DATABASE
PROJECT
DATABASE
OTHER
DataBases OTHER
DataBases OTHER
DataBases
MULTIPLE QUERIES MULTIPLE TARGETS
Multiple
Query
Sets
Multiple Target Sets Single
Metric
Single Metric
Single Metric
A G
G R
E G
A T
I O
N
MULTIPLE QUERY SETS MULTIPLE TARGET SETS
Presentation Title (acronym) ; Division – Name/OPU/HCU/BU (acronym) ; Name of Presenter ©Petroliam Nasional Berhad (PETRONAS) 2014 13
Presentation Title: DQ Metrics Architecture
Division – PETRONAS
Name of Presenter : Philip Lesslar
Presentation Title: DQ Metrics Architecture
Division – PETRONAS
Name of Presenter : Philip Lesslar
Data Quality Metrics Aggregation (practice)
Databases
of Interest
A B C D E F etc.
Data
of Interest
Data Element
of Interest
Assets
of Interest
Regions
of Interest
The (Enterprise) Dashboard
Bu
sin
ess
Fo
cu
s
Data
Fo
cu
s
The (Enterprise) Data Quality Trend
Presentation Title (acronym) ; Division – Name/OPU/HCU/BU (acronym) ; Name of Presenter ©Petroliam Nasional Berhad (PETRONAS) 2014 14
Presentation Title: DQ Metrics Architecture
Division – PETRONAS
Name of Presenter : Philip Lesslar
Presentation Title: DQ Metrics Architecture
Division – PETRONAS
Name of Presenter : Philip Lesslar
Contents
• Objectives • Requirements for global governance • Key supporting initiatives • Architecture elements • Global governance readiness • Dashboard designs • Concluding remarks
Presentation Title (acronym) ; Division – Name/OPU/HCU/BU (acronym) ; Name of Presenter ©Petroliam Nasional Berhad (PETRONAS) 2014 15
Presentation Title: DQ Metrics Architecture
Division – PETRONAS
Name of Presenter : Philip Lesslar
Presentation Title: DQ Metrics Architecture
Division – PETRONAS
Name of Presenter : Philip Lesslar
Global governance - scenarios
Central office
IQM
Metrics
engine
Country without data
export restrictions
Country with data
export restrictions
Implement default
architecture
Implement partly
distributed architecture
Our EP databases (Oracle & SQL Server)
Wells Projects Logs Our EP databases (Oracle & SQL Server)
Wells Projects Logs
Our EP databases (Oracle & SQL Server)
Wells Projects Logs
1 2 3
Presentation Title (acronym) ; Division – Name/OPU/HCU/BU (acronym) ; Name of Presenter ©Petroliam Nasional Berhad (PETRONAS) 2014 16
Presentation Title: DQ Metrics Architecture
Division – PETRONAS
Name of Presenter : Philip Lesslar
Presentation Title: DQ Metrics Architecture
Division – PETRONAS
Name of Presenter : Philip Lesslar
IQM – Architecture – Partly distributed
Central query engine runs against
central (and optionally remote)
databases
Web clients from anywhere on
the company network can
configure IQM and see results
IQM Web server and the IQM
database are centralized
Web Client
Batch engine
Database
Web forms
EP Data
Configuration/results DB
Web Client
Metrics Engine
IIS Web server
Configuration/reporting
Central Remote
User interface
Oracle Oracle Oracle Oracle Oracle Oracle
The query engine runs
in each site, under local
control
Metrics Engine
All query engines
writes metrics
back to a single
central database
Local query engine
captures metrics from
local databases
No data leaves the
country, only results
Diagram courtesy of Exprodat Inc.
Presentation Title (acronym) ; Division – Name/OPU/HCU/BU (acronym) ; Name of Presenter ©Petroliam Nasional Berhad (PETRONAS) 2014 17
Presentation Title: DQ Metrics Architecture
Division – PETRONAS
Name of Presenter : Philip Lesslar
Presentation Title: DQ Metrics Architecture
Division – PETRONAS
Name of Presenter : Philip Lesslar
Contents
• Objectives • Requirements for global governance • Key supporting initiatives • Architecture elements • Global governance readiness • Dashboard designs • Concluding remarks
Presentation Title (acronym) ; Division – Name/OPU/HCU/BU (acronym) ; Name of Presenter ©Petroliam Nasional Berhad (PETRONAS) 2014 18
Presentation Title: DQ Metrics Architecture
Division – PETRONAS
Name of Presenter : Philip Lesslar
Presentation Title: DQ Metrics Architecture
Division – PETRONAS
Name of Presenter : Philip Lesslar
DQ Dashboard designs – 4 candidate structures
DB
By Database
DB
By Datatypes
DB DB
DB
By Projects
DB
By Workflows
DB DB
1
2
3 4
Examples:
FINDER
CDS
EDM
OPENWORKS
Examples:
Well Header
Deviation
Checkshots
Examples:
Petrel Studio
Work in progress
Presentation Title (acronym) ; Division – Name/OPU/HCU/BU (acronym) ; Name of Presenter ©Petroliam Nasional Berhad (PETRONAS) 2014 19
Presentation Title: DQ Metrics Architecture
Division – PETRONAS
Name of Presenter : Philip Lesslar
Presentation Title: DQ Metrics Architecture
Division – PETRONAS
Name of Presenter : Philip Lesslar
DQ Dashboard designs – Combination 1 & 2 By Database & Datatypes
Databases Datatypes
A B C D E F G H I J K L
Presentation Title (acronym) ; Division – Name/OPU/HCU/BU (acronym) ; Name of Presenter ©Petroliam Nasional Berhad (PETRONAS) 2014 20
Presentation Title: DQ Metrics Architecture
Division – PETRONAS
Name of Presenter : Philip Lesslar
Presentation Title: DQ Metrics Architecture
Division – PETRONAS
Name of Presenter : Philip Lesslar
DQ Dashboard designs – 3, By projects
May 2014
April 2014
Presentation Title (acronym) ; Division – Name/OPU/HCU/BU (acronym) ; Name of Presenter ©Petroliam Nasional Berhad (PETRONAS) 2014 21
Presentation Title: DQ Metrics Architecture
Division – PETRONAS
Name of Presenter : Philip Lesslar
Presentation Title: DQ Metrics Architecture
Division – PETRONAS
Name of Presenter : Philip Lesslar
Concluding remarks
• Architecture is an important aspect to consider in
any implementation
• A data quality metrics initiative needs to be flexible
and scalable
• Sustainability aspects of such an initiative must
not be underestimated
• A data quality metrics program is not short term,
therefore ensure there are short term deliverables
to maintain credibility
Presentation Title (acronym) ; Division – Name/OPU/HCU/BU (acronym) ; Name of Presenter ©Petroliam Nasional Berhad (PETRONAS) 2014 22
Presentation Title: DQ Metrics Architecture
Division – PETRONAS
Name of Presenter : Philip Lesslar
Thank you