creating a measurement strategy to tell your value story

World’s largest provider of Journey Analytics solutions

Creating a Measurement Strategy to

Tell Your Value Story

November 2021Jennifer Balcom

Director of Consulting

jbalcom@explorance.com

Lan Tran

Head of Learning & Development Governance, Technology & Operations

lan.tran@kraftheinz.com

2

Head of L&D

Governance, Technology, and Operations

Hi!

I’m Lan!

7

11

Data Storytelling = Visualization + Narrative + Context

Data Visualization = Data Storytelling

“No one ever made a decision because of a number. They need a story.”

- Daniel Kahneman, November 2016

Telling Your Value Story

0

100000

200000

300000

400000

500000

600000

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

Learning Hours by Month2019 2020

2019

36Avg Learning Hours / Employee

2020

14Avg Learning Hours / Employee

2019

36Avg Learning Hours / Employee

2020

14Avg Learning Hours / Employee

2019

36Avg Learning Hours / Employee

3 hoursAvg length of course taken by

employees

12%Self-Registrations

2020

14Avg Learning Hours / Employee

1 hourAvg length of course taken by

employees

19%Self-Registrations

The outcome value story …

18

19

Challenges Benefits

Learning is rarely measured at the desired

levels to show value and alignment to

business objectives.

Learning often lacks alignment to the

business and resulting in low application on

the job.

Learning teams aren’t using data to make

decisions.

Defines true Key Performance Indicators

linked to learning effectiveness, efficiency,

value, outcomes and impact.

Identifies where learning is providing the

most value and application in the business.

Outlines roles, plans, and processes for

bringing a strategy to life through reporting,

communication, and change management.

The Need for a Measurement Strategy

Business Objective

Drive bottom-line

profitability

Drive top-line revenue

growth

Cost avoidance:

Minimize costs due to

non-compliance

Cost avoidance:

Minimize talent

acquisition/turnover

cost: “Build vs. Buy”

Leading Indicators

Productivity

Cycle Time

Cost

Scrap

Customer Satisfaction

Sales Performance

Scrap

Safety

Risk Avoidance

Manager Support

Job Application

Manager Support

Learning Effectiveness

Quality

Lagging Indicators

Profit

EBITDA

Revenue

Market Share

Legal Costs

Reputational Damage

Bench Strength

Organizational Flexibility

L&D Value Driver

Maximize Operational Efficiency

Drive Growth

Build / Maintain Foundational

Skills

Mitigate Risk$

How Does L&D Create Value? A Framework for Connecting to Business Outcomes

20

Three Areas You Should be Measuring

Effectiveness

Indicators of impact to performance and business value

▪ Learning effectiveness

▪ Content effectiveness

▪ Instructor performance

▪ Knowledge gain

▪ Application to job

▪ Business Impact

▪ Value / Net Promoter Score

▪ Time to apply on the job

Outcomes

Indicators of impact to outcomes and business results

▪ Employee turnover / retention

▪ Employee engagement

▪ Customer satisfaction

▪ Productivity gains

▪ Business outcomes impacted

▪ Critical skill gains - % of employees

▪ Career movement rate (promotion or moved

into new role with critical skill)

Efficiency & Adoption

Indicators of usage and reach across the organization

▪ Unique enrollments

▪ Average # of hours per employee

▪ Cost per employee

▪ Reach and consumption demographics

▪ Access data

▪ Top course consumption

▪ Voluntary usage

Presentation Title Subhead Can Be Placed Here

Measurement Strategy Framework

1. Establish scalable framework

2. Monitor & manage learning assets

3. Maximize L&D investments

Me

asu

rem

en

t M

atu

rity

Time

4. Demonstrate value

5. Align & partner with business

Tactical, measurement

decisions

Strategy,program

decisions

Impact to business goals

Run L&D like a business

1

2

3

4

5

Adapted from Carnegie-Mellon Capability Maturity Model The aspiration

No consistent process,Ad-hoc methods

Short Term Medium Term Long Term

Key Considerations Leadership, Governance, Culture Roles, Accountabilities, Skills Processes, Results Distribution◆ Standards, Tools, Technology

Presentation Title Subhead Can Be Placed Here

Dimensions for a Sustainable Measurement Strategy

1. Leadership: create

accountability

2. Process: integrate

measurement into work

3. Roles: establish

accountabilities and

performance goals

4. Skills / Behaviors: build

organizational muscle in

measurement and evaluation

5. Standards: establish consistent methods and measures

to streamline data collecting and reporting

7. Results Utilization:

ensure data outputs inform

decisions and actions

6. Technology: enable

the process and minimize

non-value-added work

8. Culture: establish norms

and behaviors that value

and use data

Leadership

Process

Roles

Skills

Standards

Technology

Results

Utilization

Presentation Title Subhead Can Be Placed Here

Dimensions for a Sustainable Measurement Strategy

Dimension Elements Dimension Elements

Leadership • Philosophy and strategy• Governance• Business alignment• Measurement priorities

Standards • Reporting templates• Benchmarks• Performance goals• Decision support tools

Roles • Accountability for using data for decision making has been clearly defined for all key roles Instructors, Designers, Curriculum Managers & Delivery Managers

• Business Stakeholders identified and educated

Technology • Optimized technology for:• Data collection• Analytics• Reporting • Data vizualization• Data integration

Skills / Competencies

• Analytics and interpretation• Consultation skills• Business acumen• Data visualization• Data storytelling

Results Utilization

• Improve business alignment• Enhance or retire courses• Develop instructors• Tell the value story

Processes • Front end analysis• Integrated and consistent

processes• Business rules

Culture • Data informed decisions• Success Recognition• Continuous Improvement• Funded evaluation activities

• Establish Ownership & Accountability

• Ensure High Quality Measurement Practices

• Streamline Processes And Decision Making• When to say “Yes,” “No,” or “Yes, and. . .”

• Enable Consistent Data Collection, Reporting, and Analysis

Without Continually Reinventing The Wheel

The Role of Governance

Ownerversity Measurement StrategyPLAYBOOK

TABLE OF CONTENTS

OVERVIEW….………………………………………………………………………………………………………………………………………………………………..TBD

RITUALS & ROUTINES…………………………………………………………………………………………………….……………………………………………..TBD

CHECKLISTS.…………………………………………………………………………………………………………………..……………………………………………..TBD

TEMPLATES..…………………………………………………………………………………………………………………..………………………………..…………..TBD

STANDARD EVALUATION SURVEYS………………………………………………………………………………………………………………………………..TBD

APPENDIX: DETAILED LEARNING ANALYTICS………………………………………………………………………………………………………………….TBD

Overview

Define the strategy for measuring and reporting content effectiveness and business impact

© 2020 The Kraft Heinz Company

Learning Measurement Opportunities*29

End of 2021

We are here

❖ Image from Explorance

To determine content effectiveness and drive business partnership through actionable analytics

© 2020 The Kraft Heinz Company

Kraft Heinz Measurement Strategy – V130

Demonstrate Business

Value Internally & Externally

Strengthen Business Alignment & Data Driven

Decisions

Act Like an Owner to Continuously Improve

Data Gathering & Reporting

Analytics & ImpactOutcome: Stories &

Actions

© 2020 The Kraft Heinz Company

Actionable Analytics Lead to Proactive Insights 31

Hindsight

Identify trends

Insight

Determine improvements

Foresight

Lead change

Increased engagement

Elevated performance

Improved ROI

ACTION CONTRIBUTORS

Provide data from programs All who take part in the ritual and routine

Analyze the data and create draft full report and executive summary

GTO Learning Analyst and Head of GTO

Review executive summary and full report to provide feedback

Global Learning & Development Academies, DIBs and Leadership, Chief Learning & Diversity Officer

Deliver the executive summary and present the full reportGTO Learning Analyst, Head of GTO and Chief Learning & Diversity Officer

Global Learning & Development Annual Impact Report

Ritual Objective: To communicate team accomplishments for the prior year, measure progress and business impact of learning programs, and lay out future team strategy

Frequency: Annually

When: Targeted for Q2

Ritual Length: Data gathering and analysis starts in January – April, drafts in May, reviews in June

Ritual Owner: GTO Learning Analytics

Tools/Templates: Standard Annual Report Template

Who takes part: GTO Learning AnalystHead of GTOWorkday teamPeople Analytics DIBs, Leadership and Functional AcademiesChief Learning & Diversity Officer

Data from: Google Analytics, Ownerversity, Engagement Survey, Promotion, KPI, Specific Programs (Masters, WE Lead, DIBs, Functional Academies, Ownerversity Day), Global Learning and Development Budget & Spend, Performance data (Owner 360), #LearnLikeAnOwner, industry comparison data

INPUTS

ACTIVITIES

Executive Summary delivered. Approved and published Global Learning & Development Annual Impact Report

OUTPUTS

33• Refine our Learning Measurement

Effectiveness and Analytics Strategy

• Roll out our follow-up evaluation strategy for

key learning programs

• Enhance our reporting related to functional

capabilities and diversity, inclusion, and

belonging learning experiences and their

impact

• Complete the connection from our LMS to the

Global People Analytics Data Lake

• Collaborate with our Global People Analytics

team on showing the impact learning has on

turnover, retention, promotions, sales, KPIs,

etc.

• Enhance our annual Learning Impact Report

to share all the info on how we are helping to

improve the employee experience and

positively impacting our business

What’s next?

Thank You

F O L L O W U S

www.explorance.com