managing for effective data governance: workshop for dq asia pacific congress march 2014
DESCRIPTION
This session reflects on the human aspects of Data Governance and examines what it takes to be successful in implementing effective information-enabled business transformation: * Do we need to rethink our Data Governance strategies? * Is enterprise-wide Data Management & Governance really achievable? * What techniques and capabilities do we need to focus on? * What skills and personal attributes does a Data Governance Manager need?TRANSCRIPT
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Managing for effective Data Governance Delivering value while remaining sane
Alan D. Duncan March 2014
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
“If- “ Rudyard Kipling
“If you can keep your head when all about you Are losing theirs and blaming it in you, If you can trust yourself when all men doubt you, But make allowance for their doubting too…”
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
A bit about me.... • Alan Duncan, Director of Data Governance, UNSW • 21 years Information Management & Business
Consulting – EDS, KPMG, CPW, Acuma, Pelion, SMS – Scottish Power, United Distillers, O2, Astra Zeneca,
Carphone Warehouse, Vodafone, Riyad Bank – Commonwealth Bank, NSW Roads & Maritime
Services, Centrelink, OATSIH, NSW Family & Community Services, CASA, AMSA, FaHCSIA, DAFF, Navy…
• Information-Management.com “Top 12 on Twitter” • Best supporting Actor, 2005 Barnet Drama Festival
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
…and a bit about UNSW.
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Agenda
• Do we need to rethink our Data Governance strategies?
• Is enterprise-wide Data Management really achievable?
• What techniques and capabilities do we need to focus on?
• What skills and personal attributes are needed for success?
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
PART 1. Do we need to rethink our Data Governance Strategies? Sponsored by Thomas Edison
“The value of an idea lies in the using of it.”
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
“I object!”
• “I don’t know what you’re going to do with my data once you have it.”
• “If I give you my data, you might then ask me to do some extra work to meet your additional requirements.”
• “You may not interpret the data in the same way that I do.” • “I’m an expert in this area, you’re not. The data is too
complex for you to understand.” • “It’s too difficult to get the data out of the system and I’d
need help from I.T.” • “I don’t have the budget to pay for your requirements.” • “I’d like to help but I’m just far too busy.” • “I know there are flaws in the data, but it’s good enough
for my needs. You might criticize me for the errors.” • “Management may ask additional questions and hold me
to account for the work I’m doing”.
7
“I’m not interested in preserving the status quo; I want to overthrow it.”
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Information as a Service: “True Facts” Identify measurable and targeted Business Outcomes
Why do we need information? For whom? What will we do differently?
Establish DG Operating Model
Who is accountable? By what processes?
Execute Activities & Tasks
How do we deliver? Who does the work?
Confirm the Information Holdings & Gaps
What do we need to provide? (Content + Context)
Implement DG/IMCC Services Catalogue:
What core capabilities do we need? “When it is obvious that the goals cannot be reached, don't adjust the goals, adjust the action steps.”
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Is “Open Data” a good thing?
http://www.ted.com/talks/tim_berners_lee_the_year_open_data_went_worldwide.html
9
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Outcomes of change of mindset
• Stimulus to improve data quality • Consistency of data definitions • Openness and trust • Transparency & accountability • Opportunity value • Proactive publication and Open
Data vs. “Need to know”
10
“Publish and be damned!”
http://www.ted.com/talks/tim_berners_lee_the_year_open_data_went_worldwide.html
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Summary: Rethinking Data Governance
11
• Control, structure, discipline & compliance? OR Advocacy service & information broker?
• Intimate understanding of business goals & processes
• Engagement, diagnosis & facilitation • Understand & articulate the meaning of
data, in context • Coach, mentor and advocate • Highly visible point-of-access • Self-service Information Portal • Conduit, communicate & co-ordinate • Leadership & direction • “Info as a Product”
“The art of government is to make two-thirds of a nation pay all it possibly can for the benefit of the other third.”
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Data Collection in a State Transport Infrastructure Authority, sponsored by Alfred North Whitehead
PART 1: Case Study
“The art of progress is to preserve order amid change and to preserve change amid order.”
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Data collection for road transport
• Monitoring & management of the road network
• Optimise traffic throughput • Plan for infrastructure
investment, maintenance • Incident management • Plus strategic shift from “asset
engineering” to “customer-centric” culture
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Problem: current data can’t meet modern needs • Continuing growth in traffic • Some “in-road” sensors over 30 years old
– Poor data quality – Classification by “Big, Medium, Small” – No telemetry: up to 3-month lag
• Sparse distribution of existing sensors – OK coverage in major urban areas – Few (if any) in rural areas
• Devices do “Count” only – Speed not measured
• Temporary “spot” surveys leave gaps in the record (or duplicate data!) • Over 1000 new sensors would be required
– New in-road devices approx $50K each to install (as part of road build/upgrade) – Piezoelectric “tube” devices easily damaged, poorly installed – Radar devices inaccurate in the wet
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Solution: augment with GPS vehicle tracking data
• 8000 fleet vehicles with “always on” GPS
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Benefits
• Travel time benchmarking • Flow management • Congestion “pinch point” analysis • Long-term traffic forecasting • Road safety speed zoning • Incident early-warning predictive
alerts
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Customer service: real-time information updates
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
PART 2. Is enterprise-wide Data Management & Governance really achievable? Sponsored by Confucius
“When it is obvious that the goals cannot be reached, don’t adjust the goals, adjust the action steps.”
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Information Management Strategy Drivers
Informa-on Management
Strategy
Informa-on & Data needs
Organisa-onal Strategic Direc-on
DG&IM Best Prac-ces
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Information Use Cases • Based on our current understanding of business needs, the following classes of
Information Use Case are identified • Detailed Requirements Analysis should be conducted on a project-by-project basis
to explore any detailed Use Cases within each class • Not all detailed Use Cases need to be defined ahead of time • Solutions should be flexible to accommodate new and changing Use Cases
Structured data
repor-ng
Strategic Intelligence and Data Mining
Publish content to a community
Execu-ve briefings
Educa-on, Training, Learning
Search for content previously created
Records Management, Compliance &
Audit
GIPA & Privacy
Responses
Ability to publish Filtering/screening/valida7on of what gets published
Feedback loop, measure of usefulness & con7nuous improvement Shared understanding (IT & Business)
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
To centralise or not to centralise?
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Identifying Owner & Stewards
Typically, there are significantly more unconscious Owners and Stewards
All key stakeholders in the Assets driven by an
informal structure
Business pain is felt but has no means of consistent resolution
Conscious Owners and Stewards
Responsibilities blurred and lack of understanding of the relationship
and how it should work Owners are accountable for driving up
the level of consciousness
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Summary: Information Management for the whole Lifecycle
Plan Construct,
Create, Acquire
Commission, Organise,
Store Access Use Assess Maintain Retire
• Rigorously evaluate the decision at the earliest stages of a proposal before investing in new or replacement assets.
• Manage the procurement whether it be a construction, purchase, lease or service
• Minimise the cost and risk of ownership with effective maintenance strategies and procedures.
• Manage operational costs.
• Evaluate the level of investment in assets to identify functional or physical obsolescence, financial viability, re-use opportunities and areas of unacceptable risk.
• Consult with stakeholders and plan for disposal of assets.
• Examine all options to achieve service delivery objectives and meet business requirements.
Information Owner
Chief Steward (CDO) & IMCC (cross-‐func:onal, cross domain)
Business Process
Business Process
Business Process
Business Process
Business Process
Data Stewards
An Enterprise approach to Information & Data Management requires formal organisational
processes and controls that define the rules, roles and responsibilities for information
ownership, stewardship and associated service capabilities.
Objective is to achieve explicit assurance for an agreed level of information quality
(broadest definition) and links to business value, based on the explicit capture,
formalisation and application of business rules.
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Devising the information strategy for a major retail bank, sponsored by the Vice-President of Retail Banking
PART 2: Case Study
“We must allow him to draw his sword….”
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Information Strategy for Saudi Retail Bank - Scope
• “Define Bank Data Management Best Practice - Production of a definition of Data Management Best Practice appropriate for the needs of the Bank.”
• “Education of Bank resources – education in definition of the Information Environment, Information Architecture and how Data Management fits within this.”
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
IM is business critical for Retail Banking
• Effectiveness – e.g. Marketing the right products to the right customers every time,
outperforming the competition • Efficiency
– e.g. lower effort in servicing accounts due to error reduction • Cost
– e.g. IT savings in application development and maintenance as a direct result of unambiguous information definitions
• Flexibility – e.g. Rapid and controlled ability to adapt without disruption
• Risk – e.g. Better lending decisions, more easily established Compliance,
Trust & Reputation
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Why wasn’t the Bank already doing this?
• Financial benefits are large but... – Hard to quantify (Indirect, Distributed) – Hard to realise (Contingent) – Hard to track (Causality)
• In contrast, the costs are significant and exactly quantified
• => Conventional investment appraisal is hard • => Many organisations fail to invest, and lose
competitive advantage
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Strategy Development plan – 2 phases Milestone Activity Date Description
DMD Services Solution Phase 1A started in June 17th, 2006 Assessment & Discovery
Discovery Sessions From June 18th to 26th, 2006 Over 20 interviews with key stakeholders to evaluate Bank against four major areas
of the i-Environment model. IVM Theory,Class Training June 27th, 2006
Given to DM Staff
P Senior Management Feedback Workshop for DM Solution From July 10th to 12th, 2006 Business, Technical & Architecture
Informatica Extraction Tools Training Course From July 16th to 19th, 2006 Given to DM Staff
P Detailed Plan for Phase 1B July 13th, 2006 Delivered Project Schedule P DMD Assessment Report July 30th, 2006 Delivered Document
DMD Services Solution Phase 1B started in July 15th, 2006 Detailed Solution Design
Batch Integration Inventory - Technical Analysis Sessions From July 15th to Aug. 16th, 2006 Over 25 interviews with Bank systems technical consultants, reviewing the current
batch integration architecture. P DMD Staff Profiles August 12th, 2006
Delivered Document P Batch Integration Inventory - Findings &
Recommendations August 19th, 2006 Delivered Document
P Riyad Bank Executive Management DMD Best Practice Design Presentation September 6th, 2006
This Presentation P DMD Standard Policies for Data Ownership, Data Quality,
Data Access & Data Definition Processes September 9th, 2006
Upcoming Milestone
P Implementation Plan for Phase II September 9th, 2006 Upcoming Milestone
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Strategic issues – two problems, not one
• Data problem – Accurate systematic capture, integration, distribution, storage of
granular data items
• Information problem – The common and pervasive definition, understanding, agreement of
business rules enabling consistent interpretation of data
• These issues are linked • Both need to be addressed by the Bank • The Information problem in particular is a business issue, not I.T. issue
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Key Recommendations
• RECOMMENDATION 1: Confirm the IM Vision – Share, debate & refine the vision at Exec Level – Syndicate vision across the bank management & staff
• RECOMMENDATION 2: Expand “Data Management Dept” & Position as “Information Management Dept” – Agree IMD Charter – Confirm IMD organisational structure – Locate IMD within IM Governance and Bank Organisation – Confirm organisational relationships with customers & other departments
• RECOMMENDATION 3: IM Transformation Programme – Set targets & formulate according to best practice – Plan Programme – Establish Programme & Project Governance – Release resources to the programme
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Bank IM Vision
Best Prac7ce Informa7on Management
Info at the heart of the business
Single trusted View of informa-on
Self-‐service approach for Business Info Access
An IM Func-on that challenges the Business
Build the future while suppor-ng the present
Perf. Mgnt. suppor-ng Business Strategy
Common Informa-on Model
Business data analysis, not data collec-on
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Bank IM Charter • VISION: To be the information core competency centre and champion
within the Bank, promoting the effective use of high quality information and data to maximise business value.
• DUTIES: Within the context of the Bank’s overall business Vision, Strategy and Operations, the IMD is responsible and accountable for: – Definition and assurance of Information Management Policy – Enterprise steward - looking after Enterprise Information Model & Metadata
repository – Business Intelligence centre of excellence – Information Management Services
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Implement Branch/CSR Data Capture Improvements
Review Branch/CSR Data Capture
Ini-al cycles of Data Cleansing & DQ Improvement
IMD Communica-ons
Ongoing coaching & skills transfer
Appoint/recruit ini-al IMD resources
Specific IM Skills Training Formalise IM key docs
Implementation Plan
Month 1 Month 3 Month 4 Month 5 Month 6 Month 7 Month 2
Implement IMD Capability
Mobilise IM Governance
Bank procedural changes WRT IMD Impacts
IMD Quick Wins
IM Training (IT Bas)
Implement Data Masking
Data Classifica-on Project
IMD Setup & Mobilise
Design Data Masking
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
“Data quality is like a public toilet. We all want to use it, but nobody wants to clean it.”
Vice President of Retail Banking
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
PART 3. What techniques and capabilities do we need to focus on? Sponsored by Carl Sagan
“I try not to think with my gut. If I‘m serious about understanding the world, thinking with anything besides my brain, as tempting as that might be, is likely to get me into trouble.”
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Information Asset Management
Owners
AssetManagement
Tools
Governance
Admin
Experts
User Community
InformationAsset
Steward
OwnersOwners
The “Information Asset Community”
Directives
WMSAIRS
Example High Level Data Systems & Flows
Version 1.0
General
Mandatory
Core
Corporate Support
Automatic One-way Relationship
Automatic Two-way Relationship
Manual One-way Relationship
Manual Two-way Relationship
External
External System
AIRSInterchange
AUSSAR
AME
Cyber Exams
ATSL
CLIC
SDR
Publishing System
GMEL
TRIM AD
FRLIWeb Control
Mgmt System (WCMS)
FMIS
STI
ESIR
Comweb
ATO Business
Portal
Inventory Mgmt
System (IMS)
EPK
ComBIZ Online
ProMaster
FCAT
CalumoCBMS(DoFD)
COMCARE
Thomas Logistics
SM7
HRMS
HRFlex
DTAR/OTAR
APEX
AOD Audit
AOD Case Mgmt
SystemTimelog
eRooms
Symbion Health System
API Upload System
ChangePoint Testing System
MRS
ASSP
AWS
AFD
ASIR
ADMS
TrackerATOG Job Register
AOC Surveys
Industry PaymentsCompensation Payments
Financial Actuals
Financial Actuals
Employee Expenses / Adjustment
Journals
Salary Payments
Cash Payments
Payroll (Salary)
Cash Payments / Organisation Info
Human Resources Finance
Surveillance/Audit/Reporting/Tracking
Workflow and Online Collaboration/Service Delivery
Service Delivery
Service Delivery
Service Delivery, HR & Finance, Agreements, Permissions, Aerodromes, Participants, Aircraft
Medical Examinations
Medical Exams
Surveillance /Audits/ Reporting/ Tracking
Alcohol and Other Drugs
Surveys / Surveillance
Events/Occurrences, Aircraft, Aerodromes
Surveys/ Certifications
Examinations
Work Orders
Surveillance /AuditsAerodromes
Aircraft
Events/Occurrences, Aircraft, Aerodromes
Defects/Events/Occurrences, Aircraft, Aerodromes
Exemptions Database
Alternative Means of
Compliance (AMOC)
Exemptions
AMOC / Exemptions
Human Resources – Flex
Time
Human Resources - Travel
Physical Inventory
Audit Data
workflow / service delivery
workflow / service delivery
Contacts – Ind, Org’s Contacts – Ind, Org’s
ExaminationsMedical
Examinations
Search and Rescue
Surveys
Human Resources – Time
Aircraft Equipment Finances
MMELBaseline/Minimal Equipment
Medical
PAWS
Retain
Details of Operators
Incidents
Applications / Permissions
Trending
Workflow
MAAT
Permissions / Change of Status
Permissions / Change of Status
Service Log
Alternative Means of Compliance (AMOC)
Dangerous Goods
Dangerous Goods
Content
Inventory
FTTO
FTNS
Individual Flight Data
Organisational Flight Data
Human Resources – Time
Aircraft
Individuals/ARNS
Payments
HR - TimeCash Receipts
Reconcile Invoice against
Flown Hours
Surveys / Surveillance
Enterprise Data Warehouse
CASA Internet
Airports
Landings/ Take Offs
Data
Mandatory
Core
Corporate Support
External
Business Process
Surveillance/Audit/
Reporting/Tracking
Bank Data File
PAYG payments, Salary
payments, and Superannuation
payments.
External Superannuation
Companies
Cash Payments
Superannuation Contributions
Suppliers
Remittance Advice
300+ Access Databases
Contacts
Airspace
Organisational
Human Resources
Aircraft
Permissions Info Asset Register (inventory) System Interfaces map
“Science is organized knowledge. Wisdom is organized life.”
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Analytics & Business Intelligence
37
“The alchemists in their search for gold discovered many other things of greater value.”
• “Traditional” BI (reporting & ad-hoc analysis) • Data Mining • Statistical modelling • Data visualisation • Textual analytics
• What questions do we want to answer? • What questions can we answer with the data
we’ve got? • What other data would we need? • What does the data tell us we should be
asking?
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
“Big Data” is a fact of life • Three, four, five, six “Vees”?! • A lot of data (Tb/day) • Streaming data (monitoring, flow-of-control and
alerting analytics) • Inference from semi-structured data (Twitter,
Facebook) • Synthesise insight from millions of pages of text • Programmatic analysis for specific scenarios (hard in
SQL) • A disruptive catalyst to put information at the top of
the organisational agenda • Not just about the data! Business scenarios are key • Beware the Vendors!
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
All of the data, all of the time
• Granular, forensic history • Modern data management & analytics solutions can make “all
of the data, all of the time” a reality • The bigger challenge is that the business community is not
analytically skilled enough to navigate the data and draw meaning from it…
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Get on the Cloud
40
… but security, privacy considerations are heightened.
In principle, it’s just another place to store data….
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Compliance & statutory considerations
• Freedom of Information Act 1982 (Cth) • Freedom of Information Amendment (Reform) Act 2010
(Cth) • Privacy Act 1988 (Cth) • Privacy Amendment (Private Sector) Act 2000 • Privacy Amendment Act 2012 (Cth) • Privacy Amendments (Privacy Alerts) Bill 2013 (Cth) • State Records Act 1998 (NSW) • Government Information (Public Access) Act 2009 (NSW) • Privacy & Personal Information Protection Act 1998
(NSW) • Health Records & Information Privacy Act 2002 (NSW) • NSW Government Guide To Labelling Sensitive
Information 2011 (NSW Financial & Services)
41
But is “compliance” a motivator?
“All I want is compliance with my wishes, after reasonable discussion.”
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Collaboration Culture
42
• A general willingness to share information
• Co-operative, communicative & collegiate OR control, coercion & criticism?
• The “whose data is this?” cue • Call-to-action? • Accountability & measurement?
“Respond intelligently even to unintelligent treatment.”
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Data Models & Metadata Management
Metadata Repository
Master Data Repositories UNSW Core Systems
Information Asset
Register
Physical Instantiations
Physical Layer
Logical Layer (Transition)
AnalyticalDB Models
Cubes
Conceptual Layer (Business)
Physical MessagesFormats
DWHDB
HRDB
Student Admin etc...
OperationalDB Models
Reference models
Data Subject AreasData Entities
Data Attributes
Information Concepts
Business Content Business Rules Data
Business Data Element
Domain Values
Endorsed Standards for Content
Business Constraints
Business Measures
Master data models
Classification Entity
Hierarchies
Mappings
Business Rules Definitions
Business Constraints
Business Measures
Core SystemsMDM
Metadata M
anagement Process
Information M
odel Managem
ent Process
Information Alliances: Data O
wnership & Stewardship Process
MDM Processes
Related Data Governance Processes
Application Logical Data
ModelsLogical
MessageSchemas
MDM Data Model
Systems Data Models
SOA/EP MessagesG/L
Application Logical Data
ModelsLogical
MessageSchemas
AnalyticalDB Models
Cubes
Physical MessagesFormats
OperationalDB Models
Business Glossary
Conceptual Model: Groupings & Relationships
Data Elements, Definitions, Aliases, and Security
Data Domains
Enterprise Information Model
“Do not quench your inspiration and your imagination; do not become the slave of your model.”
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Classification schemes and taxonomies
• Taxonomy: method for classification of things • Classification Scheme: grouping of kinds of
things, based on their characteristics • Information Model: representation of concepts,
relationships and semantics
• For an Enterprise approach, each should relate to the other in an ordered manner
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Information Disconnect
Careless data entry & lack of validation
Teams use different IT systems
?
Organisations change rapidly
Teams have different ways of reporting data
Month Region
Multiple codes exist for the same thing
IC_STR
Data is in different Formats
Overlapping subsets in different places
Multiple, inconsistent master data
Data Quality
“Get your facts first, then you can distort them as you please.”
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Accurate vs Inaccurate Data
Right Representation
Wrong Representation
Right Value Wrong Value
Valid Values Invalid Values
Missing Values
Accurate data Inaccurate data
“Valid” does not equal “Correct”
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Accuracy
Completeness Consistency
Integrity
Validity
Origins
Compliance
Storage Retention
Transmission Distribution Ownership
Use
Security Performance
Uniqueness
Accessibility
Flexibility
Timeliness Inherent
Pragmatic Data Quality Dimensions
• DQ Dimensions are the characteristics against which we measure quality.
• May be categorised into two types: – Inherent Quality – Pragmatic Quality
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Ongoing&BAU&opera.ons&for&Data&Governance&&&Enterprise&Informa.on&Management
Review&and&refine&Opera.ng&Model,&Processes,&Standards
KPT1305IData&Governance&
Opera.ng&ModelAccredita)ons-Simplifica)on-for-ASBQW007
Pilot-Info-Alliance-7-Student-LifecycleORG007
DG&Founda.ons DG&Enablement DG&Embedding Complete&Rollout
KPT1304&I&Consistency&of&Data&Processing&Methods
Info-Asset-Management-ProcessPROC005
Data-Owner-Role
PEOP001
Data-Interfaces-F/work-(cf-NextGen)
PROC003
Confirm-DG-Framework
STRAT007
Phase&1&
Data&Governance&Culture
Phase&2& Phase&3& Phase&4&
UNSW&Data&Governance&Strategic&Roadmap&2013/14
Data&Governance&Policy&&&Standards
Data&Governance&Processes
Data&Governance&&&Informa.on&Management&
Systems
Data&Governance&
People&&&Skills
Data&Governance&Organisa.on
DG&StrategyEndorse-DG-Charter-(Vision-&-Principles)
STRAT002
Other&Related&Projects
Confirm-the-DG-Strategy
STRAT005
DG-"Cheat7Sheet"
CUL002
DG-Org-ModelORG003
Define-Target-State-IMCC
ORG005
Pilot-Info-Alliance-7-Staff/HRORG006
Iden)fy-Data-Owners-&-Stewards
PEOP003
Core-DG-PolicyPOL001
DG-Standards-Framework
POL002
ToRs-for-Info-Alliances
ORG002
Align-EDW-Project-with-DG-Principles
SYS004
Define-DG-Strategy-for-2015+
STRAT015
Enterprise-Info-Environment-Ref.-Model
SYS008
General-Ledger-Simplifica)onQW004
Data-Steward-Role
PEOP002
DG-Communica)ons-Plan-&-Stakeholder-MappingCUL003
KPT1406&I&Improved&repor.ng&to&Government&Agencies&
KPT1404&I&Defensible&Submissions
KPT1402&I&Op.mise&ASB&Accredita.ons&Process
Pilot-Info-Alliance-7-Space-AssetsORG008
KPT1301IConfirm&DG&Scope&&&Priori.es
KPT1405&I&Improved&modelling&&&Forecas.ng
KPT1407&I&Traceable&integrity&of&Data
KPT1408&I&Streamlined&Cost&Accoun.ng
KPT1409&I&Targeted&Student&Cohorts
KPT1401&I&Space&Op.misa.on
KPT1403IIMCC&Opera.onal
Archibus-FM-Solu)onQW006
Instan)ate-IMCC-approach-7-New/Gap-capabiil)esORG010
Metadata-Management-ProcessPROC014
DQ-Dashboard-&-Repor)ngPROC002
Enabling-DG-Knowledge-Resources
POL003
Implement-Metadata/Glossary-Tools
SYS010
DQ-Profiling-&-Remedia)on-Environment
SYS009
Implement-IAM-ToolSYS002 SYS003
Enterprise-Data-Warehouse-(EDW)-Phase-1QW005
Organisa)on-Structure-Mapping-ProjectQW002
Evaluate-DQ-Logging-Tool-op)ons
SYS006
Instan)ate-IMCC-approach-7-exis)ng-capabili)esORG009
Research-Data-Storage-Ini)a)veQW003
Data-Cleanup-for-Staff-dataQW001
Select-Metadata/Glossary-Tools
SYS007
Evaluate-IAM-Tool-op)ons
SYS001
Data-Governance-Lifecycle-&-CheckpointsPROC010
Collate-ini)al-Enterprise-Informa)on-ModelPROC008
DQ-Management-ProcessPROC004
DQ-LogPROC001
EDW-Delivery-MethodologyPROC011
EDW-Design-Pa_ernsPOL004
DG-Standards-&-GuidelinesPOL005
DG-Induc)on-Training-(Owners-&-Stewards)PEOP004
Ini)al-Info-Asset-AuditPROC007
Enterprise-Informa)on-Model-as-a-ControlPROC009
Data-Governance-Lifecycle-within-SLDCPROC013
KPT1303IConsistent&Data&Defini.ons
KPT1302&I&Op.mised&data&interfaces&delivery&for&NextGen
Strategic Planning & Benchmarking
“One day Alice came to a fork in the road and saw a Cheshire cat in a tree. Which road do I take? she asked. Where do you want to go? was his response. I don't know, Alice answered. Then, said the cat, it doesn't matter.”
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Summary: Data Governance increases understanding, utility & value of information
Information System
Data Quality Management(Profiling, root-case analysis, issues tracking & resolution)
Data Modelling(Consistent, inter-operable data structures & semantic meaning)
Information Requirements & Business Analysis
(Identification & traceability of business definitions & rules)
Information Asset Register (Catalogue of data holdings)
Information SystemInformation System
Information System(s)
Data Set
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Applying the Telemanagement Forum solution models, Sponsored by Mae West
“I’m no model lady. A model’s just an imitation of the real thing.”
PART 3: Case Study
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
B2B Telco Provider (UK): the problem
• Poor lead-times for provisioning new orders • Orders fulfilled incorrectly • High levels of customer credits Caused by: • Multiple business systems • Limited levels of integration • Silo operations
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
“Order to Bill” solution approach
• Unified business processes • Enterprise Service Bus for integration • Data warehouse & BI (visibility & monitoring)
• Needed an Enterprise Information Model – quickly! – IBM model was expensive! – Company were already members of TM-Forum…
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
TMForum: communications industry trade association
www.tmforum.org
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
TMForum standard models
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
TMForum SID (Standard Information Definition)
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Mapping SID to key process groups
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Mapping SOA APIs using the cannonical model
Enterprise Service Bus (ESB)
Order Management System
Business Process Framework
Supplier Management System
Service Ac-va-on Fault Management Ra-ng/Billing
Network opera-ons & monitoring
EDW
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Data schema (example)
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Monitoring outputs
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Outcomes
• Average order times down from 3 weeks to 4 days
• Significant reduction in cancelled orders • Approx 75% reduction in customer credits
• Project was originally to take 6 months; it took over a year…
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
PART 4. What skills and personal attributes does a Data Governance Manager need? Sponsored by Mark Twain
“To succeed in life, you need two things: ignorance and confidence.”
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Forethought
• Think about both current and future demand – Cf. Google philosophy to “keep everything” – Every click, every font change…
• Imagination, innovation, entrepreneurialism • Don’t be inhibited by the current scope of existing data
“Forethought we may have, undoubtedly, but not foresight.”
Source new data; Collec-on & Integra-on; Prepara-on &
Quality. Demand-‐oriented
Inbound requests for specific requirements
Data Factory (“push”)
Product-based delivery (“pull”)
Need both “push” and “pull” modes for evidence-
based decision-making
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Expectations Management • Finding data that makes an impact • Having data for the problem at hand • Trusting the data to guide your
decision • Justifying pre-determined answers • Setting inappropriate goals • Not having the right data tools • Not thinking about value
“Two things are infinite. The universe and human stupidity. …and I’m not so sure about the universe.”
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
“That propaganda that is good which leads to success, and that is bad which fails to achieve the desired result. It’s not propaganda’s task to be intelligent, it’s task is to lead to success.”
Communication
• Listening skills – e.g. active listening
• Facilitation • Consulting & advisory • Coaching & mentoring
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
“Whosoever desires constant success must change his conduct with the times.”
Change management
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Summary: Personal attributes
66
http://www.informationaction.blogspot.com.au/2013/10/normal-0-false-false-false-en-au-ja-x_29.html
Data Owner
Data Steward
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
PART 5. Conclusions & Final Thoughts Sponsored by Terry Pratchett
“It’s still magic even if you know how it’s done.”
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Conclusions: Information Excellence EIM Framework: Enterprise Information Management Framework describes each aspect of an organisations information management state, provides a baseline of maturity against best practice and a framework of business transformation to your aspirational information management state. Provides linkage and balance between business,/IT, and human/technical aspects of EIM.
Information Governance
Information Security
Information Asset Mgmt
Metadata
Ownership & Stewardship
Information and IM Strategy and
Planning
Information and IM Quality Mgmt
Information Asset Classification
Intellectual Property
Reporting design AnalyticsInformation
Security Policy and Governance
Asset Management
Human Resources Security
Management
Knowledge Transfer
Data Mining
Data WarehousingBusiness Intelligence
Information IM Workforce
Management
Information and IM Risk Management
Registration
Data Modelling
Data management
Data Integration
Data Cleansing
Data Capture
Data Migration
Data De-duplication
Record Keeping
Knowledge Management
Information Asset Access and Use Management
Privacy Publishing
CopyrightPhysical and
Environmental Management
Communications and Operations Management
Information Security Incident
Management
Access Management
Information system
acquisition, development and
maintenance management
Compliance Management
Information and IM Policy, Principles and Architecture
Information and IM Governance Processes
Meta Knowledge
Search and Discovery
ExchangePricing
Licensing and Rights
Management
Assess and Accessibility
Redress Mechanisms Data Quality and Integrity
Data Conversion & Transformation
Record Management Archiving Conservation and Preservation
Record Creation and Capture
Digital Continuity
Collection Management
Retrieval and Access
Retention and Disposal
Business Continuity
Enterprise Informa7on Model IM Solu7ons and Technology IM Policies
Organisa7on and People Data Governance Informa7on Culture
IM Processes
Business Processes
DB Models
Definitions, Derivations, Decision Rules, Execution Rules
IM Governance Process
IM Stewardship Process
Technical MetaData Management
Logical Model
ETL SpecsReport
DefinitionsSemantic Specs
Data Marts
ETL Cubes
SemanticLayer
Standard R
eport Library
ETLOperational System Staging Warehouse
Conceptual Model
Logical Model
Physical Model
Capture & FormaliseRequirements
& Rules
Impact Assessment
& Implementation
MetadataLineageImpact
Etc.
MetadataCollection
Asset Alignment/Mgt
Architecture Changes
Architecture Mgt
A holistic, data-centric approach to Information Management & Data Governance, addressing both human and technical factors in both Business and IT domains
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Recap: Seven Transformational Levers • Strategy (are the vision, objectives and overall direction for the
organisation clearly articulated and well understood? Do Business Strategy, IT Strategy and Information Strategy align?)
• Culture (is the desired behaviour exhibited throughout the organisation?) • Organisation (are the organisational structures appropriate to executing
the Strategy?) • People (is the workforce properly skilled and motivated?) • Process (do all business processes align with and support the Strategy?) • Policy (are the organisational controls appropriately defined and applied?) • Systems (does the infrastructure of IT Systems provide the right support
for all key business processes?)
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Calls to action
• WHAT: Identify 3 key issues (data related) that need to be addressed within your business, and the IM capability areas that support these
• WHY: Outline the business outcomes/benefits that you would derive from addressing these issues
• HOW: Map these changes to the Seven Transformational Levers of the Enterprise IM Framework
See example worksheet
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Decision Request / Action Plan - 6-step method
1. REQUIREMENT: State one key issue (data related) from the strategic list. What is needed?
2. PROBLEM/INHIBITOR: what is currently preventing your organisation from doing something about it?
3. OUTCOME: What specific benefits would your organisation derive if this was addressed?
4. SOLUTION: What new product or capability is needed to deliver the requirement stated in (1)?
5. PLAN: Outline the step-by-step action plan & timescales that will deliver the outcome.
6. DECISION REQUEST: In THREE bullet points, state what specific support you need from your Sponsor in order to get things started
– “I need you to agree to… 1, 2, 3.” – e.g. budget, resources, new policy, key communication
See example worksheet
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
FINAL THOUGHTS: perspectives on Information Management
Applica-ons/Systems Architecture (How do I access it?)
Data Architecture (Conceptual, Logical, Physical – What does it mean?)
Informa-on Asset Catalogue (What have we got? In what context? For whom?)
Info Management & Data Governance Roadmap (When will it be delivered?)
IM Business Case (Why do we want it? How much will it cost?)
Business Process Models (How do we do it?)
Business Services Framework (What do we do?)
Business Lifecycle (Why do we do it?)
Business IM Capability & Transforma7on (Who is accountable?)
IM Capability
Assessment (What do we want/
need?)
IM Service Capability /
IMCC
(Who delivers & supports it?)
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Blog links…
1. RETHINKING OUR STRATEGIES: http://informationaction.blogspot.com.au/2012/07/information-as-service-what-is-it-and.html
2. IS EIM ACHIEVABLE?: http://informationaction.blogspot.com.au/2013/11/to-centralise-or-not-to-centralise-that.html
3. CAPABILITIES: http://informationaction.blogspot.com.au/p/the-information-management-tube-map.html
4. SKILLS & QUALITIES: http://informationaction.blogspot.com.au/2013/10/normal-0-false-false-false-en-au-ja-x_29.html
5. MAGIC: http://informationaction.blogspot.com.au/2014/03/now-thats-magic.htm
73
Alan Duncan, Director of Data Governance, UNSW E: [email protected] Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan
Uncontrolled when printed
Intellectual curiosity Skeptical scrutiny Critical thinking
http://www.informationaction.blogspot.com.au/
@Alan_D_Duncan
http://www.linkedin.com/in/alandduncan