measuring impact - tying learning to strategic business outcomes

Measuring Impact: Ways to connect corporate learning to strategic business outcomes

Housekeeping

#LearningImpact

@Danij2 @CarolLeaman

@Axonify @Bersin

Event is being recorded

Please leverage the Q&A window

Dani Johnson

Vice President, Learning and Development Research

Bersin by Deloitte

Deloitte Consulting LLP

Carol Leaman

Chief Executive Officer

Axonify

Copyright © 2016 Deloitte Development LLC. All rights reserved. 4 Measuring Impact: How to Connect Corporate Learning to Strategic Business Outcomes

Who we are Global provider of leading practices, trends, and benchmarking research in talent management, learning, and strategic HR.

6 research practices •  Human Resources

•  Leadership Development

•  Learning & Development

•  Talent Acquisition

•  Talent Management

•  Tools & Technology

Offerings

•  WhatWorks® Membership: Research, Tools, Education, Consulting

•  IMPACT: The industry’s leading conference on the Business of Talent

•  Advisory Services & Consulting

Human Resources

Talent Acquisition

Leadership Development

Learning & Development

Talent Management

Tools & Technology


•  Why is learning measurement important, especially now?

•  How do we improve learning measurement– including connecting learning to business outcomes?

•  Case Study – measurement in practice.

Agenda

Learning measurement is becoming more important… but L&D often isn’t good at it.


Measuring learning is becoming more important.

Source: High-Impact Talent Analytics, Bersin by Deloitte, 2013 State of Learning Measurement, Bersin by Deloitte, 2015; Knowledge Advisors: The Business Drivers of Talent Analytics, Knowledge Advisors, 2014

Business leaders are looking to L&D for data insights.

51%

Percent of surveyed leaders asking for human capital

data




81%

77%

58%

56%

15%

Finance

Operations

Sales

Marketing

HR

Other departments use data to make decisions.


51%


data




81%

77%

58%

56%

15%

Finance

Operations

Sales

Marketing

HR

Other departments use data to make decisions.


51%


data

Research indicates that better measurement ties to

better performance.

Alignment Efficiency

Effectiveness


Only five percent of L&D organizations we surveyed excel at using data to:

5%

•  Align with the business, •  Efficiently run the L&D function, •  Increase the effectiveness of

learning methods.

Source: State of Learning Measurement, Bersin by Deloitte, 2015


Help improve learning measurement by:

Transforming data into decisions.

Create partnerships and

access to data

Aligning measurement with business priorities.

Build and maintain relationships with stakeholders and understand

how they define success

Measuring business Metrics.

Focus practices, resources and

tools on measuring business – not busy - matters

Aligning measurement with business priorities.


Focusing on business (not learning…) outcomes. …which are typically reflected in learning business plans

69%

58%

48%

38%

35%

27%

19%

Budget

Program investment priorities / plans

Staffing

Volume (e.g. learners, hours)

Business impact goals

Customer satisfaction targets

We do not have a business plan for L&D.



Building ongoing relationships with stakeholders. …L&D should create and maintain alignment with business needs


77%

64%

42%

40%

Meet with line managers and business leaders to diagnose problems and develop solutions

Enlist executives or business leaders as sponsors

Hold regular reviews with a steering committee of business and HR leaders

Require line managers and business leaders to sign off on new learning initiatives


Building ongoing relationships with stakeholders. …Learners are stakeholders too!

2%

3%

2%

3%

10%

9%

14%

15%

20%

24%

11%

16%

20%

23%

25%

32%

34%

40%

36%

39%

31%

33%

38%

35%

39%

38%

34%

30%

28%

21%

56%

48%

40%

39%

26%

21%

18%

15%

16%

16%

Not important Somewhat important Very important Essential

Sharing knowledge with my team

Web search for resources

Professional networks / communities

External news feeds and blogs

Content curated from external sources

Self-directed study / courses

Internal company documents

Performance support tools

Company training / e-learning

General meetings and conversations

Source: Learning in the Workplace Survey, Centre for Learning & Performance Technologies / Jane Hart,, 2014

Focusing on measuring business metrics.


Using metrics typically considered valuable by the business.

0%

25%

50%

75%

100%

0% 25% 50% 75% 100%

Valu

able

Tracked / Measured

Impact on behavior / job performance

Sponsor / client satisfaction

Business results

Impact on talent / organizational metrics

Line manager satisfaction / feedback

Content access methods

Learning scores / transfer

Learner demographics

Hours delivered / time spent learning

Costs / expenses Learner

satisfaction / feedback

Enrollments, starts, usage, and completions



Thinking holistically about measurement.

Outcome (the business)

Efficiency (L&D)

Alignment (managers /

leaders)

Effectiveness (learners)

• Time to develop • Cost to develop / deliver • Cost of development per minute / hour • Cost per participant • Resource hours to develop / deliver • Total enrollments

•  Learning offerings correspond to competency model •  Learning has strong executive sponsors •  Managers support participation among team

members •  Program objectives align with business needs

• New behaviors reflect what was introduced in a learning offering

• New knowledge and skills • Learner satisfaction • Adoption

• Key performance indicators (KPIs) • Revenue • Profitability and growth • Market share • Customer / satisfaction • Employee engagement / retention • Productivity



Measuring against established business objectives. Frequency with which L&D groups evaluate themselves against…


16%

22%

28%

59%

28%

7%

9%

7%

21%

18%

19%

10%

Satisfaction targets

Progress toward strategic initiatives

Volume targets (e.g., programs, hours delivered)

Sales performance or targets

Weekly/Monthly After learning events Rarely/Never

Transforming data into decisions.


Applying the basics.

Dedicate resources and manpower 11% Of companies surveyed spend

more than 5% of their budget on measurement. 28% don’t even know.

Build relationships with stakeholders 1.2x Companies that regularly engage

with stakeholders are more likely to have high-impact measurement capabilities.

Have a plan 19% Of those we surveyed said their companies don’t have a learning business plan for L&D.

Source: State of Learning Measurement, Bersin by Deloitte, 2015; Becoming a High-Impact Learning Organization, 8/2012


Kirkpatrick method is only the beginning

Source: Key Trends in the Measurement of Talent Development, 3/2012

Rethinking yesterday’s tools and practices.

Too simplistic…

Does liking a course or passing a test really translate directly into better performance or business results?

Too hard…

There are no clear, consistent definitions and it’s rarely simple or easy to isolate learning contribution (…or L&D’s).

And too backward

Finding out after training that people were not satisfied – or that results didn’t improve – doesn’t tell you what to change.


Utility / usefulness

Stakeholder satisfaction

Experience API (xAPI) Employee engagement

Learning records stores Readiness / promotions

Information technology Adaptive learning Employee retention

Finance and accounting Web analytics Time to proficiency

Marketing Social listening Activity / contributions

Business intelligence CRM systems Influence and feedback

New friends New tools New metrics

Building (some) new capabilities.


Keep business priorities at the forefront

Learning measurement may require a new mindset.

• Relationships • Learning business plan

• Frequent measurement

• Holistic approach • Identify who will benefit

• Practices, resources, capabilities, & tools

Actionable data through: • Partnerships • Data transparency

Focus Align Transform

Case Study: Measurement in practice.

Case Study: Large Retailer

• Over 75,000 employees

• Currently focused on logistics and distribution network

• Over 150 facilities across the U.S.

• Over 400,000 learning sessions (1.4M Questions) per Month

• 1 Million Safety Observations recorded per Month

Assess � Predict � Adjust � Optimize

Assess Hope


Large Retailer


Did the number of safety incidents

meet expectations?

Large Retailer


Did they actively handle the materials

correctly?


meet expectations?

Large Retailer

How do we measure the behaviors?


Did they gain knowledge on

correct materials handling?


correctly?


meet expectations?

Large Retailer

How do we measure knowledge?


Material Handling Learning Modules and Reinforcement (Pull Hook & Layer

Picking)




correctly?


meet expectations?

Large Retailer

How do know if it’s the right content?

High level of knowledge

Correct behavior demonstrated Good content

High level of knowledge

Incorrect behavior

demonstrated

Content may be an issue


Material Handling Learning Modules and Reinforcement (Pull Hook & Layer

Picking)




correctly?


meet expectations?

Large Retailer

Dani Johnson

Vice President, Learning and Development Research

Bersin by Deloitte

Deloitte Consulting LLP

Carol Leaman

Chief Executive Officer

Axonify

Q&A

Thank you!

About Deloitte As used in this document, “Deloitte” means Deloitte Consulting LLP, a subsidiary of Deloitte LLP. Please see www.deloitte.com/us/about for a detailed description of the legal structure of Deloitte LLP and its subsidiaries. Certain services may not be available to attest clients under the rules and regulations of public accounting.

This communication contains general information only, and none of Deloitte Touche Tohmatsu Limited, its member firms, or their related entities (collectively, the “Deloitte Network”) is, by means of this communication, rendering professional advice or services. Before making any decision or taking any action that may affect your finances or your business, you should consult a qualified professional adviser. No entity in the Deloitte Network shall be responsible for any loss whatsoever sustained by any person who relies on this communication.

Copyright © 2016 Deloitte Development LLC. All rights reserved

Member of Deloitte Touche Tohmatsu Limited

About Axonify

Axonify focuses on driving employee knowledge in corporate enterprise, ensuring that every employee has what they need - in their heads or at their fingertips - to effectively do their jobs.

For the first time ever, organizations can measure the impact of their learning efforts on knowledge growth through to employee behaviors and business results.

A Little Bit About Axonify

Axonify counts amongst its customers the world’s largest beverage company, the world’s largest retailer, the world’s largest automotive company, the world’s largest professional services firm, the world’s largest pharmaceutical company and the world’s largest multinational conglomerate.

10 M 79%

>80% 78%

Microlearning bites per month and growing

Voluntary participation rates

“Axonify has made me more knowledgeable”

“Axonify has made me more effective at my job (i.e. work safer, sell more)”

Axonify Delivers Axonify Users Say Source: 2014 Axonify End User Survey

Some of Our Customers

Visit www.axonify.com to learn more