Download - Enabling an Analytics-Driven Organization
The First Step in Information Management
www.firstsanfranciscopartners.com
Enabling an Analytics-Driven Organization
Kelle O’Neal
415-425-9661
@1stsanfrancisco
Samra Sulaiman
202-320-9764
pg 2
Why We’re Here
Purpose:
Understand the People, Process and Technology needed to support an Analytics-Driven Organization
Outcome: Understanding how Data Management and Data Governance Support Analytics Knowing the organizational constructs needed to trust Analytics Data An ability to manage change Practical knowledge that can be immediately implemented
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
Table of Contents
1. Introduction: Clarification of terms and Level setting 2. Enabling Analytics through EDM:
• Master and Reference Data Management • Meta Data Management • Data Quality • Architecture • Security and Privacy
3. Creating “line of sight” from Data to Analytics 4. Building the Organization 5. Addressing Change 6. Findings from Research: The relationship between Descriptive and Predictive
Analytics 7. Summary and Wrap up
pg 3 © 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
What is Analytics?
Data Insight Action
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential pg 4
Our Focus Today
pg 5
The Big Picture: EIM Framework
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
Provides a holistic view of information in order to manage information as a corporate asset
Enterprise Information Management
Information Strategy
Architecture and Technology Enablement
Content Delivery
BI, Performance Management , and Analytics
Data Management Information Asset Management
GOVERNANCE
ORGANIZATIONAL ALIGNMENT
Content Management
Why is EIM Important?
Internal pressures:
Desire to understand customer at any time from any channel
Data Quality issues are persistent
Balance of old mainframe systems with new technologies
Movement to the cloud and losing control of data
Data Volumes are increasing
Mobile apps enabling data to be created and accessed anywhere
Project oriented approach to addressing issues/opportunities
External pressures:
Greater amounts of new regulations
Increasing Customer Demands – my information anywhere at any time
Technology and market changes outpacing ability to respond
EIM ensures the right people are involved in
determining standards, usage and integration of data across projects, subject areas and lines of
business
pg 6 © 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
Develop and execute architectures, policies and procedures to manage the full data lifecycle
Enterprise Data Management
Enterprise Data Management Ensure data is available, accurate, complete and secure
Data Quality Management
Data Architecture Data
Retention/Archiving
Master Data Management
Big Data Management
Metadata Management Reference Data Management
Privacy/Security
DATA GOVERNANCE
pg 7 © 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
Develop and execute architectures, policies and procedures to manage the full data lifecycle
Symbiotic Relationship
An EIM initiative is an important component of a Data Governance
Strategy
Must Have “Tools”…
• Documented and enforced governance policies and processes
• Clear accountability, ownership and escalation mechanisms
• Continuous measurement and monitoring of data quality & adoption
• Executive support to create a culture of accountability around the quality of the data…it’s everyone’s concern
• Solid alignment between business & IT
Technology alone will not solve the problem
You can’t “do” EIM without Data Governance
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential pg 8
Data Governance Definition
Data Governance is the organizing framework for establishing the strategy, objectives and policy for effectively managing corporate data.
It consists of the processes, policies, organization and technologies required to manage and ensure the availability, usability, integrity, consistency, auditability and security of your data.
Communication and Metrics
Data Strategy
Data Policies and Processes
Data Standards
and Modeling
A Data Governance
Program consists of the inter-workings
of strategy, standards, policies
and communication
pg 9 © 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
pg 10
Data Governance Framework
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
• Vision & Mission • Objectives & Goals • Alignment with Corporate
Objectives • Alignment with Business
Strategy • Guiding Principles
• Statistics and Analysis • Tracking of progress • Monitoring of issues • Continuous Improvement • Score-carding
• Policies & Rules • Processes • Controls • Data Standards & Definitions • Metadata, Taxonomy,
Cataloging, and Classification
• Operating Model • Arbiters & Escalation points • Data Governance Organization
Members • Roles and Responsibilities • Data Ownership & Accountability
• Collaboration & Information Life Cycle Tools
• Data Mastering & Sharing • Data Architecture & Security • Data Quality & Stewardship
Workflow • Metadata Repository
• Communication Plan • Mass Communication • Individual Updates • Mechanisms • Training Strategy
• Business Impact & Readiness • IT Operations & Readiness • Training & Awareness • Stakeholder Management & Communication • Defining Ownership & Accountability
Change
Management
How Data Management / Governance facilitates Analytics
Provides a focus on data as a foundational asset of the company so that it can be used in multiple ways effectively
Defines data standards to ensure data consistency
Maps data from source to target and identifies transformations
Creates rules, standards, policies and processes for data cleansing and validation
Articulates most trusted and timely data sources to facilitate data sharing
Identifies potential data irregularities and creates a process to resolve them
“Between 25 percent and 30 percent of a BI initiative typically goes toward initial data cleansing.”
Competing on Analytics, Davenport and Harris
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential pg 11
www.firstsanfranciscopartners.com
Enabling Analytics Through EDM
• Master and Reference Data Management • Meta Data Management • Data Quality • Architecture • Security and Privacy
Develop and execute architectures, policies and procedures to manage the full data lifecycle
Enterprise Data Management
Enterprise Data Management Ensure data is available, accurate, complete and secure
Data Quality Management
Data Architecture Data
Retention/Archiving
Master Data Management
Big Data Management
Metadata Management Reference Data Management
Privacy/Security
DATA GOVERNANCE
pg 13 © 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
Develop and execute architectures, policies and procedures to manage the full data lifecycle
Develop and execute architectures, policies and procedures to manage the full data lifecycle
Enterprise Data Management
Enterprise Data Management Ensure data is available, accurate, complete and secure
Data Quality Management
Data Architecture Data
Retention/Archiving
Master Data Management
Big Data Management
Metadata Management
Privacy/Security
DATA GOVERNANCE
pg 14 © 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
Develop and execute architectures, policies and procedures to manage the full data lifecycle
Reference Data Management
pg 15
Master Data Management (MDM)
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
What is MDM?
MD is a technology-enabled discipline in which business and IT work together to ensure the uniformity, accuracy, stewardship, semantic consistency and accountability of the enterprise’s official shared master data assets. Master data is the consistent and uniform set of identifiers and extended attributes that describes the core entities of the enterprise including customers, prospects, citizens, suppliers, sites, hierarchies and chart of accounts.
pg 16
MDM Key Considerations
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
Category Decision
Entity Types
• What type of data will be managed in the MDM Hub • What are the agreed upon definitions of each type • What is the required cardinality between the entity types • What constitutes a unique instance of an entity
Key Data Elements • Purpose, definition and usage of each data element
Hierarchies and Relationships • Purpose, definition and usage of each hierarchy / relationship structure
Audit Trails and History • How long do we have to keep track of changes
Data Contributors
• What type of data do they supply • Why is this needed • At what frequency should they supply it • What should be taken for Initial load versus ongoing
pg 17
MDM Key Considerations (continued)
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
Category Decision
Data Quality Targets • How good does the data have to be • Root cause analysis
Data Consumers • Who needs the data and for what purpose • What do they need and at what frequency
Survivorship • What should happen when…
Lookups • Which attributes are lookup attributes • What are the allowable list of values per attribute • How different are the values across the applications and how do we deal with
inconsistencies
Types of Users and Security • What types of users have to be catered for • Can they create, update, delete, search • Can they merge, unmerge
Delete • How should deletes be managed
Privacy and Regulatory • Privacy and regulatory issues
Develop and execute architectures, policies and procedures to manage the full data lifecycle
Enterprise Data Management
Enterprise Data Management Ensure data is available, accurate, complete and secure
Data Quality Management
Data Architecture Data
Retention/Archiving
Master Data Management
Big Data Management
Metadata Management Reference Data Management
Privacy/Security
DATA GOVERNANCE
pg 18 © 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
Develop and execute architectures, policies and procedures to manage the full data lifecycle
pg 19
Reference Data Management
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
What is Reference Data?
Reference Data enables an enterprise to make sense of its data and to turn it into real business information. Reference Data are those codes that categorize data and enable an organization to compare that data with internal and external sources.
What is the Total sales for all Males who are Silver Customers that live in states on the Eastern Seaboard and are 35-44 years old?
Develop and execute architectures, policies and procedures to manage the full data lifecycle
Enterprise Data Management
Enterprise Data Management Ensure data is available, accurate, complete and secure
Data Quality Management
Data Architecture Data
Retention/Archiving
Master Data Management
Big Data Management
Metadata Management Reference Data Management
Privacy/Security
DATA GOVERNANCE
pg 20 © 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
Develop and execute architectures, policies and procedures to manage the full data lifecycle
pg 21
Meta Data Management
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
Ente
rpri
se
Goal: A common glossary governed at the enterprise level
Lin
e o
f B
usi
nes
s
Application A
Business Glossary includes:
• Common terms, definitions, business rules, etc.
Conceptual Data Model or Enterprise Logical Data Model includes:
• Key business concepts/subject areas
• Key business relationships
Application B
Data Model/Dictionary Data Model/Dictionary
Model to Model
Develop and execute architectures, policies and procedures to manage the full data lifecycle
Enterprise Data Management
Enterprise Data Management Ensure data is available, accurate, complete and secure
Data Quality Management
Data Architecture Data
Retention/Archiving
Master Data Management
Big Data Management
Metadata Management Reference Data Management
Privacy/Security
DATA GOVERNANCE
pg 22 © 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
Develop and execute architectures, policies and procedures to manage the full data lifecycle
pg 23
Data Quality
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
What is Data Quality?
The planning, implementation and control activities that apply quality management techniques to measure, assess, improve and ensure the “fitness of data” for use.
pg 24
Data Quality Dimensions
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
Dimension Key Questions Impact
Completeness Is all appropriate information readily available? Are data values missing or in an unusable state?
Incomplete data can cause major gaps in data analysis which results in increased manual manipulation and reconciliation
Conformity Are there expectations that data values need to reside in specified formats?
If so, do all values conform to those formats?
By not maintaining conformance to specific data formats, there is an increased chance for data misrepresentation, conflicting presentation results, discrepancies when creating aggregated reporting, as well as difficulty in establishing key relationships
Consistency Is there conflicting information about the same underlying data object in multiple data environments?
Are values consistent across all data sources?
Data inconsistencies represent the number one root cause in data reconciliation between different systems and applications. A significant amount of time by business groups is being consumed with manual manipulation and reconciliation efforts
Accuracy Do data objects accurately represent the “real-world” business values they are expected to model?
Incorrect or stale data, such as customer address, product information, or policy information, can impact downstream operational and analytical processes
Uniqueness Are there multiple, unnecessary representations of the same data objects within a given data set?
The inability to maintain a single representation for each entity, such as agent name or contact information (across all component business systems), leads to data redundancy and inconsistency, as well as increased complexity in terms of reconciliation
Integrity Which data elements are missing important relationship linkages that would result in a disconnect between two data sources?
The inability to link related records together can increase both the complexity and accuracy of any corresponding business intelligence derived from those sources. It directly correlates to the level of trust the business has in the data
Timeliness Is data available for use as specified and in the time frame in which it was expected?
The timeliness of data is extremely important. Data delayed in data denied. Could lead to reporting delays, providing slate information to customers and making decisions based stale data
Develop and execute architectures, policies and procedures to manage the full data lifecycle
Enterprise Data Management
Enterprise Data Management Ensure data is available, accurate, complete and secure
Data Quality Management
Data Architecture Data
Retention/Archiving
Master Data Management
Big Data Management
Metadata Management Reference Data Management
Privacy/Security
DATA GOVERNANCE
pg 25 © 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
Develop and execute architectures, policies and procedures to manage the full data lifecycle
pg 26
Reference Architecture: Conventional EDW
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
Data Quality Business Rules Engine Meta Data Management Security/Privacy
Acquisition Management Aggregation/ Persistence
Access/Delivery
Staging (structured
data)
• Cleanse • Enrich • Transform • Create
golden record (MDM)
Sources
ODS
Data Mart(s)
Analytics
Data Services
Other Data Retrieval Systems
Archival services
EDW • Internal: HR, Finance
• External: Market data, Credit scores
pg 27
Reference Architecture: How Big Data Fits In
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
Data Quality Business Rules Engine Meta Data Management Security/Privacy
DATA GOVERNANCE
Acquisition Management Aggregation/ Persistence
Access/Delivery
Staging (structured
data)
• Cleanse • Enrich • Transform • Create
golden record (MDM)
Sources
ODS
Data Mart(s)
Analytics
Data Services
Other Data Retrieval Systems
Archival services
EDW
Structured: • Internal: HR,
Finance • External:
Market data, Credit scores
Unstructured: • Sentiment • Clickstream • PDF
Semi-structured: • XML, JSON
Staging (Semi-structured & unstructured data)
Hadoop
Develop and execute architectures, policies and procedures to manage the full data lifecycle
Enterprise Data Management
Enterprise Data Management Ensure data is available, accurate, complete and secure
Data Quality Management
Data Architecture Data
Retention/Archiving
Master Data Management
Big Data Management
Metadata Management Reference Data Management
Privacy/Security
DATA GOVERNANCE
pg 28 © 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
Develop and execute architectures, policies and procedures to manage the full data lifecycle
Security
pg 29 © 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
Confidentiality
Integrity Availability
According to NIST, “The security management practices domain is the foundation for security professionals' work and identifies key security concepts, controls, and definitions. NIST defines computer security as the "protection afforded to an automated information system in order to attain the applicable objectives of preserving the integrity, availability, and confidentiality of information system resources (this includes hardware, software, firmware, information/data, and telecommunications).”
Your Analytics infrastructure and data should comply with the normal InfoSec and Privacy practices of your organization!
pg 30
Key Security Domains
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
1. Security and Risk Management: Security, Risk, Compliance, Law, Regulations, and Business Continuity
2. Asset Security: Protecting Security of Assets
3. Security Engineering: Engineering and Management of Security
4. Communication and Network Security: Designing and Protecting Network Security
5. Identity and Access Management: Controlling Access and Managing Identity
6. Security Assessment and Testing: Designing, Performing, and Analyzing Security Testing
7. Security Operations: Foundational Concepts, Investigations, Incident Management, and Disaster
8. Software Development Security: Understanding, Applying, and Enforcing Software Security
Reference: (ISC)2
pg 31
Big Data Analytics Privacy
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
According to U.S. President’s Council of Advisors on Science and Technology, “Big data drives big benefits, from innovative businesses to new ways to treat diseases. The challenges to privacy arise because technologies collect so much data (e.g., from sensors in everything from phones to parking lots) and analyze them so efficiently (e.g., through data mining and other kinds of analytics) that it is possible to learn far more than most people had anticipated or can anticipate given continuing progress.”*
Some Key Challenges include:
Difficulty in data anonymization and masking due to sheer volume, number of sources and variety of data
Collecting information without explicit consent
Lack of or insufficient data governance practices – according to Rand Secure Archives Data Governance Survey in 2013, “44% of the respondent have no formal data governance policy”*
Infrastructure – both on-premise and cloud-based
Reference: MIT Technology Review Custom + Oracle Courtesy of Samra Sulaiman, ConsultData, LLC
pg 32 © 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
Analytics
pg 33 © 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
What happened?
Why did it happen?
What will happen?
How can we make it happen?
Diagnostic
Prescriptive
Descriptive
Predictive
Reference: Gartner
Val
ue
Difficulty
Courtesy Samra Sulaiman, ConsultData, LLC
Key Components of an Effective Analytics Strategy
People Process
Technology Data
pg 34 © 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
pg 35
People
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
Roles Key Responsibilities
Business SME/Manager (decision maker) • Defines the business problem, business objectives
Analyst (explorer) • Specific domain area expert • Works with raw data • Creates reports • Leverages data visualization tools
IT • Sets up infrastructure • Pre-processes data • Tests and deploys models
Data Scientist • Develops models • Performs statistical analysis • Explores data trends, anomalies
Process
Descriptive
Prescriptive
Prescriptive
pg 36 © 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
Courtesy of Samra Sulaiman, ConsultData, LLC
Typical Analytics Life-cycle:
• Define the business problem—e.g., forecasting future sales based on past performance
• Design data requirements—e.g., aside from internal data sources, can data be enriched using external data sources such as credit scores, social media data feeds?
• Pre-process data—rationalize and cleanse data; apply the appropriate level of data quality dimensions
• Perform data analytics—data analytics can be performed using various algorithms or machine learning techniques to gain insight
• Visualize the results—various tools can be leveraged to visualize the insight, show anomalies, etc.
Define Problem
Design
Pre-process
Analyze
Visualize
pg 37
Technology
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
Define your use cases before selecting your tools!
www.firstsanfranciscopartners.com
Creating “Line of Sight”
• Select use cases: How EDM Impact Analytics
pg 38
From Data to Analytics
Data Insight Action
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential pg 39
pg 40
Select Use Cases
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
How EDM Impacts Analytics:
1. How Data Quality impacts Analytics:
Demand forecasting
Sentiment analysis
2. How Meta Data impacts Analytics:
Glossary
Lineage
3. How Master Data Management impacts Analytics:
Hierarchy management
pg 41
How Data Quality Impacts Analytics
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
Demand forecasting: a retail company wants to forecast future product sales based on historical data to better manage inventory
Data Quality Considerations: Accuracy Completeness Consistency Timeliness Uniqueness Integrity Conformity
Focus: highest quality data Courtesy of Samra Sulaiman, ConsultData, LLC
pg 42
How Data Quality Impacts Analytics (continued)
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
Sentiment analysis: a software product company wants to analyze consumer feedback quickly after product launch
Focus: relevant data - eliminate ‘noise’ quickly Courtesy of Samra Sulaiman, ConsultData, LLC
Data Quality Considerations • Accuracy • Completeness Consistency Timeliness • Uniqueness • Integrity • Conformity Relevance (new)
pg 43
How Meta Data Impacts Analytics
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
• How data flows across the infrastructure/company?
• How the data is derived? • How is data transformed?
• Data type changes • Calculations • Business/DQ rules
What? Why? How?
• Ability to track upstream/data producers and
downstream/data consumers • Which system transformed the data? • How data was transformed (which rules and calculations
were applied)? • Ability to perform root-cause-analysis – tracing data errors
from a report back to the source
• What data exists today? • Who owns the data? • Which system is the ‘System of
Record’ or ‘Trusted Source’? • Are there standard business rules
for that data?
• Common understanding of available data • Ability to locate needed data more quickly • Ability to know who can answer questions about the data • Ability to trust the data due to the governance process • Audit trail of who touched/changed a term • Data quality rules, metrics, etc.
pg 44
How MDM Impacts Analytics
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
Global Company
US
Subsidiary A
Branch A Branch B Branch C
Subsidiary B
Europe
Subsidiary C
Branch D Branch E
Business Challenges:
Regulatory Compliance - e.g., inability to uniquely identify all counterparties to a transaction
Sales & Marketing - e.g., inability to roll-up sales by subsidiaries or by region
Product – e.g., poor inventory management due to lack of product hierarchy and inconsistent product data
pg 45
How MDM Impacts Analytics (continued)
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
Executive sponsorship is critical!
Business-driven with close collaboration with IT
Holistic strategy to avoid re-work later; leverage existing (funded) projects, if possible
Strong Data Governance is key to success
Iterative process – rapid and continuous delivery of key capabilities that business needs
pg 46
A Balancing Act!
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
Rank Number Topic As a …. I want to… So that I can … Acceptance Criteria
1 16 “Project” Field
identifier
Data
Governance
Lead
identify from which project the Work In Progress
business terms has originated and from which
project the Work In Progress business term groups
has originated. Work In Progress is part of
Ungoverned Business Terms.
see which project teams are accountable
for specific business terms and be able to
manage them accordingly during the work
in progress business term section
A “free-form” text field titled “Project ” that can be completed on
the same page as the business term and business term group is
being developed in the InfoMap tool.
Target Release Date: 05/01/15 (IT)
Note:-Managing this via business term group can be complicated
since some of the business terms are shared by multiple projects.
This approach has been rejected.
2 1 Data Concept
Management in
InfoMap
CG Data
Governance
Lead
- be able to see all DG defined core, non-core (i.e. BL
governed) & ungoverned business concept with their
associated data subject and data concept
association (per the Core Concepts spreadsheet)
mange the core and non-core business
concepts
Flags - CG core / non core, BL ownership (IM, DIST, SERV, GBS and
other entities e.g. PCS, ITG, GBS HR, GBS FIN, etc.),
Standard report configuration that can be shared between the CG
DGL & B/L DGLs.
3 2 Data Concept
Management in
InfoMap
CG Data
Governance
Lead
- For all core & non-core assign the accountable,
consulted and informed data governance business
l ine & named owner
manage the ongoing definition, assignment
of accountability of Data Subjects and
Concepts within InfoMap
Report status and progress
ACI assignment at the BL level.
Standard report configuration that can be shared between the CG
DGL & B/L DGLs.
4 3 Data Concept
Management in
InfoMap
CG Data
Governance
Lead
- see core & non core (business governed) data
concepts that have no business definition by
business l ine and owner.
assign and manage the development of
core concept definitions to their
appropriate owners.
Add Inflight (WIP) Group within Governed Business Terms
Standard report configuration that can be shared between the CG
DGL & B/L DGLs.
Data Sources DQ Solutions Your Data Management
solutions should be proportionate to your Data Analytics needs and focused on business value
Courtesy Samra Sulaiman, ConsultData, LLC
www.firstsanfranciscopartners.com
Data Governance Operating Models
pg 47
Data Governance is critical for Analytics
pg 48
You can’t “do” Analytics without Data Governance
An Analytics initiative is an important use case for a Data
Governance Office
Must Have “Tools”…
• Documented & enforced data quality policies and processes to ensure data consistency and standards
• Understood business logic that maps data from source to target
• Clear data accountability, ownership and escalation mechanisms
• Continuous measurement & monitoring of data quality, adoption & value
• Clearly defined data elements, attributes and computation/derivation of shared data
• Really know your data quality before diving into an Analytics “Project”
Data Governance is the program that ensures that the Analytics
content is trusted
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential pg 48
Operating Model
Outlines how Data Governance will operate
Forms basis for the Data Governance organizational structure – but isn’t an org chart
Ensures proper oversight, escalation and decision making
Ensures the right people are involved in determining standards, usage and integration of data across projects, subject areas and lines of business
Creates the infrastructure for accountability and ownership
Wikipedia: An Operating Model describes the necessary level of business process integration and data standardization in the business and among trading partners and guides the underlying Business and Technical Architecture to effectively and efficiently realize its Business Model. The process of Operating Model design is also part of business strategy.
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential pg 49
Process to create an Operating Model
• How are decisions made?
• Who makes them?
• How are Committee’s used?
Culture
• Centralized
• Decentralized
• Hybrid
Operating Model • Data Governance
Owner
• SME’s
• Leadership
People
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential pg 50
Pros: Formal Data Governance
executive position
Data Governance Steering Committee reports directly to executive
Data Czar/Lead – one person at the top; easier decision making
One place to stop and shop
Easier to manage by data type
Cons: Large Organizational Impact
New roles will most likely require Human Resources approval
Formal separation of business and technical architectural roles Bus / LOBs
Operating Model - Centralized
DG Executive Sponsor
DG Steering
Committee
Center of Excellence (COE)
Data Governance Lead
Technical Support
Data
Architecture
Group
Technical
Data
Analysis
Group
Business Support
Business Analysis Group
Data Management
Group
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential pg 51
LOB/BU Data Governance Steering Committee
LOB/BU Data Governance Working Group
pg 52
Operating Model - Decentralized
Data
Stewards
Application
Architects
Business
Analysts Data Analysts
Pros: Relatively flat organization
Informal Data Governance bodies
Relatively quick to establish and implement
Cons: Consensus discussions tend to take
longer than centralized edicts
Many participants compromise governance bodies
May be difficult to sustain over time
Provides least value
Difficult coordination
Business as usual
Issues around co-owners of data and accountability
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
pg 53
Operating Model - Networked
Pros: Flat structure Data management is aligned to lines of business and/or
IT ensuring there is a clear understanding of data requirements for that organizational unit
Relatively quick to establish and implement Known documented connections and RACI charting
creates accountability without impacting an organization chart
Cons: Collaborative decisions tend to take longer to
implement than centralized edicts
Many participants compromise governance bodies (making it potentially unruly)
RACI’s and the Network itself needs to be maintained
Little enforceable consistency around managing data across the enterprise
Difficult coordination Autonomy at the LOB level can be challenging to coordinate
Data Governance Office
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
pg 54
Operating Model - Hybrid
Pros: Centralized structure for establishing appropriate direction
and tone at the top
Formal Data Governance Lead role serving as a single point of contact and accountability
Data Governance Lead position is a full time, dedicated role – DG gets the attention it deserves
Working groups with broad membership for facilitating collaboration and consensus building
Potentially an easier model to implement initially and sustain over time
Pushes down decision making
Ability to focus on specific data entities
Issues resolution without pulling in the whole team
Cons: Data Governance Lead position is a full time, dedicated role
Working groups dynamics may require prioritization of conflicting business requirements
Too many layers
Data Governance Steering Committee
Data Governance Office
Data Governance Working Group
Business Stakeholders IT Enablement
Data Governance Organization
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
pg 55
Operating Model - Federated
Pros: Centralized Enterprise strategy with decentralized
execution and implementation
Enterprise Data Governance Lead role serving as a single point of contact and accountability
“Federated” Data Governance practices per Line of Business (LOB) to empower divisions with differing requirements
Potentially an easier model to implement initially and sustain over time
Pushes down decision making
Ability to focus on specific data entities, divisional challenges or regional priorities
Issues resolution without pulling in the whole team
Cons: Too many layers
Autonomy at the LOB level can be challenging to coordinate
Difficult to find balance between LOB priorities and Enterprise priorities
Enterprise Data Governance Steering Committee
Enterprise Data Governance Office
Data Governance Groups
Data Governance Organization
Business Stakeholders
IT Enablement
Divisional DG Office
Business Stakeholders
IT Enablement
Divisional DG Office
Business Stakeholders
IT Enablement
Business Stakeholders
IT Enablement
Divisional DG Office
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
Operating Model Roles and Responsibilities
Data Governance Steering Committee
− Provides overall strategic vision
− Approves funding, budget and resource allocation for strategic data projects
− Establishes annual discretionary spend allocation for data projects
− Adjudicates intractable issues that are escalated
− Ensures strategic alignment with corporate objectives and other business unit initiatives
Data Governance Office
− Chairs the Data Governance Steering Committee and Data Governance Working Group
− Acts as the glue between the Data Governance Steering Group and the Working Committee
− Defines the standards, metrics and processes for data quality checks, investigations, and resolution
− Advises business and technical resources on data standards and ensures technical designs adhere to data architectural best practices to ensure data quality
− Adjudicates where necessary, creates training plans, communication plans etc.
Data Governance Working Group
− Governing body comprised of data owners across Business and IT functions that own data definitions and provide guidance & enforcement to drive change in use and maintenance of data by the business
− Validates data quality rules and prioritize data quality issue resolution across the functional areas
− Trains, educates, and creates awareness for members in their respective functional areas
− Implements data business processes and are accountable to decisions that are made
pg 56
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
www.firstsanfranciscopartners.com
Aligning Governance and Analytics
pg 57
Example: Data & BI Governance Structure
Accountable for Governance and Change Leadership for Data & BI across Company • Executive Data & BI Owner
• Forum Chair
• Membership – Executive Process Owners
• Meeting Cadence - Monthly
Data & BI Governance Leadership Forum
Accountable for Master Data Quality across (Customer, Product, etc) • MDM Practice Lead • Membership – Chief Data Stewards
• Meeting Cadence – Bi-Weekly
Data Stewardship Forum
Accountable for BI Standardization & Adoption • BI Practice Lead
• Membership – Functional Reporting Leads • Meeting Cadence – Bi-Weekly
Business Intelligence Forum
Strategy & Guidance Agreed Decisions
Strategic Initiative Alignment
Initiative Requests Project / Initiative Progress Intractable Issues
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential pg 58
Example: Data & BI Governance Forum Roles
Forum to strategize, plan and review Master Data initiatives led by team members Forum to drive Company’s Performance Measurement Architecture development Forum to discuss & define strategic direction impacting policy, process & technology Data & BI decision forum for Project X as well as other Corporate initiatives
Market to Sell
Idea to Offer
Finance World Wide Operation
Hire to Retire Issue to
Resolution to Prevention
FP&A IT
Data & BI Governance Leadership Forum
Forum Chair – Data Governance Sr. Director
Process Owner VP SALES SVP Product Strategy CFO SVP Services VP HR SVP Finance VP Info Tech
Data Ownership Customer Product Chart of Acct Vendor Employee
Executive Data & BI Owner – EVP XXX
Executive Process Owner’s represent the Functions within their Process Domains • Active participation is critical to our success • When necessary, delegation to peer Functional Owners is acceptable
VP WWOPS
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential pg 59
Example: Analytics Operating Model – Immediate
IT Advisor
Enterprise Infrastructure Committee
Executive Office (CEO)
Strategy & Risk
(CRSO)
IT (CIO)
Accounting (CAO)
Global Services (COO)
PMO
Head of Business Analytics
CEO
Credit Analytics Client Analytics Market Analytics
LOB … LOB … LOB …
Data Stewards
Executive Sponsor Analytics CFO
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential pg 60
Example: Analytics Operating Model – Long Term
IT Advisor
Enterprise Infrastructure Committee
Executive Office (CEO)
Strategy & Risk
(CRSO)
IT (CIO)
Accounting (CAO)
Global Services (COO)
PMO
Executive Sponsor Analytics CFO
Head of Business Analytics Analytics Working Group
LOB Reporting LOB … LOB … LOB … LOB … LOB …
CEO
Credit Analytics Client Analytics Market Analytics
LOB … LOB … LOB …
Analytic Directors
Data Stewards
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential pg 61
Example: Data Governance and Analytics
Sponsor - Data Governance
Business Steward Leads
Data Governance Office DGO Chair
IT Lead DG Coordinator
Data Management IT Support Group
Data Governance Working Group
Data Stewards
Marketing Fin. Accting
Fin. Treasury Risk ECM Ops.
HR
Fin. FP&A
Credit Admin Fin. Ext. Reporting
Legal/ Compliance
SVB Analytics Privacy/CSO
IT Advisor
Enterprise Infrastructure Committee
Executive Office (CEO)
Strategy & Risk (CRSO)
IT (CIO)
Finance (CAO/CFO)
Global Services (COO)
PMO
Executive Sponsor Analytics CFO
Head of Business Analytics
Analytics Working Group
Analytic Directors
Credit Analytics
Client Analytics
Market Analytics
LOB … LOB … LOB …
CEO This is a role / relationship chart and NOT an organization chart
Data Stewards
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential pg 62
Example: Data Governance and Analytics
Sponsor - Data Governance
Business Steward Leads
Data Governance Office DGO Chair
IT Lead DG Coordinator
Data Management IT Support Group
Data Governance Working Group
Data Stewards
Marketing Fin. Accting
Fin. Treasury Risk ECM Ops.
HR
Fin. FP&A
Credit Admin Fin. Ext. Reporting
Legal/ Compliance
SVB Analytics Privacy/CSO
IT Advisor
Enterprise Infrastructure Committee
Executive Office (CEO)
Strategy & Risk (CRSO)
IT (CIO)
Finance (CAO/CFO)
Global Services (COO)
PMO
Executive Sponsor Analytics CFO
Head of Business Analytics
Analytics Working Group
Analytic Directors
Credit Analytics
Client Analytics
Market Analytics
LOB … LOB … LOB …
CEO This is a role / relationship chart and NOT an organization chart
Data Stewards
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential pg 63
www.firstsanfranciscopartners.com
OCM Basics
pg 64
EIM Means Change
Successful EIM means a change to the information management culture, processes and policies
Changing that culture means that you are asking people to think and behave differently about how data is accessed and used
You need an organized and systematic way to manage and sustain those changes – or there is marginal likelihood of success
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential pg 65
Two Sides to Change Management
WHO? WHAT? WHEN? WHERE? WHY?
Something old stops, and something new starts
Relatively easy to plan for and anticipate
SITUATIONAL REORIENTATION PEOPLE GO THROUGH AS THEY COME TO TERMS WITH THEIR NEW
SITUATION It’s important to help affected individuals let go
of the old situation and get comfortable with the new way
Everyone processes at a different rate and are rarely aligned with the milestones of the implementation plan
PSYCHOLOGICAL
For change to be successful, BOTH sides need to be addressed © 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
pg 66
Why Do People Resist Change?
Loss of identity and their familiar world
− Loss Analysis
Disorienting experience of the transition between the old and the new
Weak/no sponsorship by executive leaders and managers
− Lack of alignment
− No involvement
Overloaded with current responsibilities
No answer to WIIFM
No involvement in the crafting the solution
Each individual’s capacity to handle change
Other work and personal issues
How well an organization has handled changes in the past
67 © 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
pg 67
68
What Might Resistance Look Like?
Trying to outlast the changes: bargaining for exemption from new policies or processes
Reduction in productivity and missed deadlines
Going back to the old way of doing things
Lack of attendance in project status meetings and events, or training
Higher absenteeism
Open expression of negative emotion
From executives, resistance could be:
− No visible sponsorship of data governance; no open endorsements
− Refusal or reluctance to provide needed resources and/or information
− Repeatedly canceling or refusing to attend critical meetings
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential pg 68
69
Getting People through Change Successfully Requires….
Clear definition of what is changing
− Make sure the new behaviors, skills and attitudes are clearly defined and communicated
− Provide examples, training and allow time for practice
Attention to feedback:
− What are people saying and how are you addressing it?
− You must respond to feedback; be sure and attach the actions you take to the feedback you received so that associates know you are listening
Some reward or recognition structure to encourage new behaviors
Measurement and performance management
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential pg 69
Communication Framework to Drive Change
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
State the aspiration, the BHAG (Big, Hairy, Audacious Goal) What is the desired
future? What is the value of
the future state to the company? How does that
future state move forward the overall business? High-level, strategic
statement of a Goal
Vision Picture Plan Participation Paint the picture of
how the future will look and feel once Data Management is implemented. How are people
going to get their work done and interact with each other? How will a day be
organized? Future State Principles
Lay out the plan for achieving the future state; the steps and timeline in which people will receive information, training, and the support they need to transition to the future
Orient managers to tell employees how and when their worlds are going to change
Start with where people are and work forward
What does this mean to me?
Overall Roadmap Group specific
roadmaps
Establish each person’s part in both the future state and the plan to get there
Show associates their roles and relationships to each other in the future
Show associates what part they play in achieving the future and the transition process to get there
All this helps them let go of the past and focus on the future
What is my role? Who does What Across Enterprise Group Specific
Adapted from William Bridges, Managing Transitions
Explain why we’re doing what we’re doing - the purpose behind the outcome What is/was the
problem? Who said so and on
what evidence? What could occur if
no one acts to solve it? What could happen
if that occurs? Why you are
executing your Vision
Purpose
pg 70
Change Management Phases
Organizational alignment implemented Structure Jobs/people Policies/procedures Incentives Performance management
Change integration/adoption assessment
Communication plan execution Training development and delivery Feedback and analysis of results Leadership alignment checkpoint Measurement approach & metrics Organizational impact analysis Resistance management Implementation checklist development
Information gathering and analysis Stakeholder Analysis Sponsorship development Change plan development Leadership alignment checkpoint Communications planning Training needs assessment and planning
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential pg 71
Meet with Program/Project Manager, and lay out CM Approach for the Program/Project
Monitor & Refine Extend
Change Management Alignment to EIM Phases
****Communication Launch
Information Gathering and Analysis
Stakeholder Analysis/Loss Analysis
Change Readiness Assessment
Leadership Alignment
Sponsorship Development: Assessment and Road Map
Detailed Change Planning
Communication Plan
Operationalize Implement Strategize & Plan Assess & Align Project Initiation
Planning for Change
****Collect, Analyze and Report on Feedback
Implementation Checklist
More Leadership Alignment
Metrics and measurement
Org Impact Analysis: structure, jobs, training, policies
Managing Change
****Lesson Learned Assessment
Organizational Alignment Action Plan
Change Integration Checklist
Transitioning to the Business
Implementing/ Sustaining Change
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential pg 72
The Potential Value of OCM to Your Business
Can result in real monetary value to the business
− Acceleration of planned changes
− Faster realization of planned benefits
− Minimizes business disruption: loss of staff, lower productivity, etc.
Greater likelihood that the IM/DG changes implemented will be sustained
pg 73 © 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
The Correlation is There
There is data that shows a strong correlation between effectively managing change and meeting objectives
− “Show me the numbers”
Analysis from:
− Prosci’s Best Practices studies
−McKinsey studies
− Your own organizational experience?
pg 74 © 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
2002 McKinsey Study
Examined 40 projects
Evaluated
− ROI expected
− ROI delivered
− Level of change management effectiveness
Results Direct correlation between change
management effectiveness and gap between ROI expected and ROI delivered
Those that were above average on those
factors realized 143% of expected value
Those that were below on all three factors
realized 35% of expected value From the article “Helping Employees Embrace Change”, McKinsey Quarterly 2002 Number 4, Jennifer A. LaClair and Ravi P. Rao
pg 75 © 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
pg 76 © 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
Using reorganization as an opportunity to change
mind-sets and behaviors of the workforce
Focusing as much on how the new organizational
model would work as on what it looks like
Accelerating pace of implementation to make the new
model deliver value as soon as possible
Addressing all risks and bottlenecks as early as
possible, before and during implementation
Developing a clear communication plan for all
internal and external stakeholders
Ensuring that IT, financial, human resources, and
other systems were updated to support new
organizational model
Defining detailed metrics for reorganization’s
effect on short and long-term performance
and assessing progress against them
KEY STRATEGIES KEY PROCEDURES
2010 McKinsey Reorganization Study
Taking Organizational Redesign from Plan to Practice, McKinsey Global Survey Results, 2010 Courtesy of McKinsey & Company
Contributes to: Achieving Project Objectives…
pg 77 © 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
Consider:
There is a direct and profound effect that a strong change management program bears on an organization’s ability to meet or exceed its project objectives
95% of those who rated their change management program as excellent met or exceeded their project objectives as opposed to only 17% of those who rated their change management program as poor or non-existent
pg 78 © 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
pg 79
Contributes to: Staying on Schedule…
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
pg 80
Consider:
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
There is also significant correlation between the quality of the change management program and the project’s ability to stay on or ahead of schedule.
75% of those respondents with excellent change management programs had projects that were on or ahead of schedule
Poorly Managed Change Results in:
Lower productivity
Resistance
Turnover of valued employees
Apathy for the future state
Arguing about the need for change
More people taking sick days or not showing up
Changes not fully implemented; benefits not achieved
People finding work-arounds or reverting to the old way of doing things
The change being totally scrapped
Divides are created between ‘us’ and ‘them’
pg 81 © 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
pg 82
Bottom Line…
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
There is significant correlation between the effectiveness of a change management program
and achieving Data Governance results
Five Key Factors for DG Success
Executive Sponsorship
Aligned leadership
Clear communication (early and often)
Stakeholder Engagement
Measurement
83 © 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
Final Thoughts
Be absolutely clear and specific about what is changing and what that will mean in terms of required behavior changes: people can’t change behavior if they don’t know what they’re supposed do differently.
Appreciate that there is a psychology to change: understanding how people react is essential to structuring your initiative to deal with it.
84 © 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
www.firstsanfranciscopartners.com
Survey Results
pg 85
Descriptive Analytics Program
86 © 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
• Descriptive Analytics/BI is still going strong
Predictive Analytics Program
87 © 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
• Predictive Analytics is still emerging
Relationship of the Programs
88 © 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
• Majority are under the same umbrella
• Very little outsourcing of overall Program
Predictive Analytics Investment
89 © 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
• Big Market Opportunity
Descriptive Analytics Investment
90 © 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
• It’s not “either/or”, both are receiving investment
Organizational Construct
91 © 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
• All Analytics will be managed together
Measuring Effectiveness of Descriptive Analytics
92 © 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
• Improved data is a measurement of effectiveness
• Usual suspects of better decision making, better understanding of results, improved processes
Measuring Effectiveness of Predictive Analytics
93 © 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
• Very similar to Descriptive Analytics
Relative Measures of Effectiveness
94 © 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
• No comparison between the programs, although they appear to be seeking a similar outcome
Take-Aways
• Organizations still struggle for skilled resources Resources
• Descriptive BI isn’t going away Investment
• Optimization needs to occur across Descriptive and Predictive BI Outcome
pg 95 © 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
www.firstsanfranciscopartners.com
Wrap-Up
pg 96
Approach to practical, affordable analytics
Identify efficiency and operational metrics for BI Analytics environments
Confirm scope and seed Analytics &
Metrics model
Define cost of ownership and operating standards
Synthesize and map to
benchmarks
Develop and present efficiency
sustaining plan
Rationalize metrics and predictive models
Align metrics to business
Develop transition plan to unified
metrics
Data Efficiency Corporate Metrics
Analytics / BI Architecture Sustaining
Plan
pg 97 © 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
98
Kelle O’Neal
415-425-9661
@1stsanfrancisco
© 2015 First San Francisco Partners www.firstsanfranciscopartners.com Proprietary and Confidential
Thank you!
Samra Sulaiman
202-320-9764
www.firstsanfranciscopartners.com
Who Took the Survey?
0.0% 5.0% 10.0% 15.0% 20.0% 25.0% 30.0%
Executive Management
Finance Management and / or Reporting
Content and / or Digital Asset Management
Application Development
Data and / or Information Architecture
Business Intelligence and / or Analytics
Information / Data Governance
Corporate Research and / or Library
Marketing and / or Market Research
IT Management
Software or System Vendor
Other
Job Function
Demographics
Demographics
Demographics
Demographics
Business Intelligence v Data Science
Business Intelligence v Data Science
Business Intelligence v Data Science