research brief - · pdf file©2015 brandon hall group. licensed for distribution by...

Research Brief

Measuring the ROI of Informal Learning

Brandon Hall Group Research TeamDecember 2015

©2015 Brandon Hall Group. Licensed for Distribution by Docebo. 2


TABLE OF CONTENTSIntroduction 3

Understanding the Framework 4

PuttingitintoPractice 6

Measuring the Informal 8

Key Takeaways 12

AuthorsandContributors 13

About Brandon Hall Group 14



NOTABLE INSIGHT

IntroductionOrganizations are finally realizing that people learn more of what they need to be effective at their job through in-formal channels, on-the-job experiences and coaching than they do through more formal means. When it comes to developing and executing the learning strategy, how-ever, companies continue to look at things completely upside-down. The vast majority of the learning delivered within organizations is through formal classrooms and e-learning courses, which only accounts for about 10% of the learning that occurs. Even within that 10%, retention rates for single, formal learning events are abysmal, with most learners forgetting close to 90% of what they learned over time. What opportunities are companies missing to help people retain more of what they are learning and discover new knowledge through other, more informal channels?

Source: 2014 Brandon Hall Group, Social and Collaborative Learning Study (n=254)

Figure 1 Expected Frequency of Learning Interactions

24%

37%

29%

9%

1%

DailyormoreOften

Weekly

Monthly

Annually

Lessthanannually

The vast majority of the learning delivered within organizations is through formal class-rooms and e-learning courses, which only ac-counts for about 10% of the learning that

occurs.



NOTABLE INSIGHT

Understanding the FrameworkThe 70/20/10 model has been discussed for more than two decades, yet companies are just now beginning to realize the realities of the model and what it means for them. It isn’t necessarily a prescription that mandates companies keep formal learning to 10% of their entire scope of learning programs. Instead, the model lays out a reality of how people learn on the job, no matter what the learning function does. This means companies need to realign their learning strategy to leverage, promote and facilitate the informal and experiential learning that is occurring within the organization. When we look at the actual mix of learning that occurs within organizations that have fully adopted the framework, we can see that it is not a one to one ratio:

Source: 2015 Brandon Hall Group 70/20/10 Study (n=248)

Figure 2 Mix of Learning in 70/20/10 Organizations

As you can see, learning efforts are split pretty evenly be-tween formal and on-the-job learning, with a smaller mix of informal learning. Companies can and should expect to devote more than 10% of their efforts on formal learning. This is the type of learning and content that requires this kind of guidance and vision from the organization. But or-ganizations should also plan to enable the other types of

The 70/20/10 mod-el It isn’t necessarily a prescription that man-dates companies keep formal learning to 10% of their entire scope of learning programs. Instead, the model lays out a reality of how people learn on the job, no matter what the learning function

does.



NOTABLE INSIGHT

learning rather than letting them occur on their own and hope for the best. By approaching the blend this way, or-ganizations can begin to measure the informal and expe-riential learning that occurs.

The other key consideration in adopting the framework is its execution. It is not something designed to be a sequen-tial rollout, where the formal class is delivered, some in-formal learning occurs, then the learner fills in the gaps on the job as they go. Instead, it is a blended, cyclical rollout where parts of the classroom or course experience are collaborative and informal, then the work environment (where the new skills and behaviors are demonstrated) includes collaborative elements such as coaching and for-mative, on-the-job learning.

Figure3 Blended 70/20/10 Rollout

A great example of this in practice is the “flipped class-room,” where learners do pre-work such as watching videos or reading resource material before to the formal classroom event. They can also communicate and discuss the content ahead of time in collaborative workspaces. When they arrive to the event, they already have an un-derstanding of the material and can begin collaborating on scenarios and problem solving, with the instructor act-ing more as a facilitator. When the event is over, learners have a much better understanding of the material and how to apply it than if they had simply attended

Source: Brandon Hall Group 2015

The 70/20/10 frame-work is not something designed to be a se-quential roll out, where the formal class is de-livered, some informal learning occurs, then the learner fills in the gaps on the job as they

go.



NOTABLE INSIGHT

a class. After the class, they can continue to discuss and share perspectives as they put the skills and behaviors to use on the job.

Putting it into PracticeAccording to Brandon Hall Group’s 2015 Learning & Development Strategies Study, the number one driver for developing a learning strategy is to align L&D goals with the goals of the business. In other words, the outcome of learning needs to be performance and not simply learning itself. Impacting the business is the biggest ROI learning can have. As it turns out, organizations that have adopted the 70/20/10 framework into their learning strategies are indeed seeing returns in that area.

Source: 2015 Brandon Hall Group 70/20/10 Study (n=248)

Figure 4 Benefits of the 70/20/10 Framework

By incorporating more informal learning, companies say they are better able to make that crucial link between learning and how well their people perform and how well the organization performs. Other tangible benefits com-panies say they have reaped from the framework include accelerated leadership development, increased use of in

By incorporating more informal learn-ing, companies say they are better able to make that crucial link between learning and how well their people perform and how well the organization per-

forms.



NOTABLE INSIGHT

formal learning, improved team performance, and opti-mized use of learning technologies.

Brandon Hall Group research has found that there are a few key learning characteristics that are correlated with high-performing organizations. When we look at organi-zations that have annually improved KPIs such as reve-nue, market share and customer satisfaction, we see that they:

• Support proactive learners -- employees who take re-sponsibility for their own development and seek out learning.

• Provide easy-to-use tools that present employees with opportunities to connect to formal and informal knowledge sources throughout the organization.

• Present each and every employee with opportunities for development based on personal strengths, weak-nesses, job roles, or interest from sources that go be-yond their immediate leaders.

These characteristics are indicative of organizations where informal learning is embraced, facilitated and leveraged. These findings add to the mountain of research that says that organizations that focus more on the 70 and the 20 tend to perform better.

There are obviously many elements to the framework that have varying degrees of effectiveness. Not surpris-ingly, one of the most effective elements of the frame-work is instructor-led classroom training. Remember, the framework is not about eliminating formal learning, but rather making the most of it. Some of the other more ef-fective elements include:

Remember, the frame-work is not about elim-inating formal learn-ing, but rather making

the most of it.



NOTABLE INSIGHT

• On-the-job assignments

• Peer-to-peer learning

• Coaching/mentoring

• Short-term projects

• Exposure to managers

Measuring the InformalOverall, organizations are not great at measuring learning. Once you get beyond completion rates and smile sheets, measurement gets somewhat complex. Because it can be difficult to understand the impact that a learning program has had on the organization or determine the ROI, com-panies simply don’t do it. In fact, only half of organizations measure more than 75% of their programs at Kirkpatrick Level 1 -- learner reaction. The best practice recommen-dation from Kirkpatrick for Level 1 is 100%. So, we are not even doing the easy stuff very well. The story gets worse when we look at measuring beyond formal learning.

Figure 5 Effectiveness of Learning Measurement

Just 2% of organizations say they are highly effective at measuring informal learning, and half say they aren’t at all effective. The fact that companies are having such a

Source: 2015 Brandon Hall Group Learning Strategies Study (n=302)

14%

37%30%

19%

Formal

43%

16%

32%

7%

Experimential

Notatall Moderately Somewhat Highly

51%

17%

30%

2%Informal

The fact that compa-nies are having such a hard time measuring informal learning may be a major factor in why they aren’t deliv-ering as much of it as

they could or should.

Experiential InformalFormal



NOTABLE INSIGHT

hard time measuring informal learning may be a major factor in why they aren’t delivering as much of it as they could or should.

A big reason why there isn’t a lot of measurement of the informal is that companies just don’t have the tools at their disposal to do it. When we asked organizations what aspects of their LMS platform they liked least, the third least-liked was reporting features, and the fourth was a lack of analytics features to determine ROI. In fact, ROI analytics scored an average of 2.7 on a satisfaction scale of 1 to 5.

If companies cannot measure the formal learning they are doing, which has systems, standards and platforms ded-icated to measurement, how can they hope to measure the learning that is taking place outside of the confines of the formal learning machine? The first step in measuring the ROI of informal learning is to get a strategy in place for measuring learning in general.

The measurement strategy should be an extension of a learning strategy focused on performance as an outcome. If this link is established, measurement becomes clear. In fact, for the most part, measuring informal learning is not that different than measuring more traditional types of learning. There are the basics:

• How many people participated in the program?

• Did they like it?

• How did they do on the assessments after partici pating?

Then, the performance measurements:

• Are people better at their jobs after participating?

• Are learners getting to competency or optimal productivity faster since the informal learning program was initiated?

The first step in mea-suring the ROI of infor-mal learning is to get a strategy in place for measuring learning in general. The measure-ment strategy should be an extension of a learning strategy fo-cused on performance

as an outcome.



While measurement of the informal needs to follow the same principles as more traditional learning measure-ment, there are also informal-specific measurements as well:

• Which learners are participating the most?

• Who do people turn to most often for information?

• What content is getting shared the most?

• What are people searching for?

These metrics give an organization insight into how learn-ers are using the informal tools as well as how effective they are. In fact, when learners are able to rate and com-ment on content, you get instant Level 1 feedback that is not only useful to the learners, but can be used to shape content in the future.

Ideally, you need to determine the benefits derived from informal learning (either one type or several used to-gether) and assign a value to each. That is, of course, the most difficult part. It can be just as difficult to determine the costs of these informal learning endeavors. But if an organization ever wants to seriously calculate the ROI of informal learning, it must be done. The truth is, if

Figure6 Informal Learning Metrics

INFORMAL SPECIFIC

Source: Brandon Hall Group 2015



NOTABLE INSIGHT

organizations are already calculating the ROI of learning in general, they know to make these cost/benefit deter-minations. If they are not, then they shouldn’t be worried about the ROI of informal learning, either.

Once these cost and benefit values are determined, ROI becomes (relatively) simple math:

• There is the cost-benefit analysis where ROI = (Benefits/Costs) x 100.

• Or, one can use the alternative where ROI = (Benefits - Costs)/Costs) x 100.

• There’s also a forecasted calculation where ROI =(Benefits - Costs)/Costs) x 100 x (%A) x (%P), where %A is the percentage of learners expected to participate and %P is the percentage of learners who do participate.

In the end, making an informal tool or experience avail-able to learners should impact their learning. If they are learning better, faster, or more, that is reflected not only in learning outcomes, but in performance outcomes as well.

The truth is, if orga-nizations are already calculating the ROI of learning in gen-eral, they know to make these cost/ben-efit determi nations. If they are not, then they shouldn’t be worried about the ROI of infor-

mal learning, either.



Key Takeaways• Re-examine your organization’s learning strategy through the lens of 70/20/10. Are you providing oppor-tunities for learners to engage in informal, collaborative and experiential learning? If not, it’s time to think about how to incorporate some new learning environments.

• Find the blend that works for your organization. Don’t get wrapped up in getting down to 10% formal learn-ing. Just make sure that the material you are delivering in these classes and courses might not be better served via multiple delivery types. It is what modern learners ex-pect and what they need to retain and execute on newly learned skills and behaviors.

• Make sure you have a sound overall measurement strategy before you get too worried about informal mea-surement. Then you can explore what unique measure-ments informal learning offers, as well as technologies like Tin Can API (xAPI) that can track informal learning the way an LMS can track formal events.



Authors and ContributorsDavid Wentworth ([email protected]) is Principal Learning Analyst at Brandon Hall Group. He has been a senior research analyst in the human capital field since 2005, previously with the Institute for Corporate Productivity and joining Brandon Hall Group in early 2012.

Claude Werder ([email protected]) is the Vice President of Research Operations and Principal HCM Analyst at Brandon Hall Group. His responsibilities include overseeing Brandon Hall Group’s team of analysts, direct-ing research priorities, content quality assurance, and producing the annual HCM Excellence Conference.

Nissa Benjamin ([email protected]) is the Marketing Coordinator at Brandon Hall Group. She is responsible for graphic design, social media and webinar management.



About Brandon Hall GroupBrandon Hall Group is a HCM research and advisory services firm that provides insights around key performance

areas, including Learning and Development, Talent Management, Leadership Development, Talent Acquisition

and HR/Workforce Management.

With more than 10,000 clients globally and 20 years of delivering world-class research and advisory services,

Brandon Hall Group is focused on developing research that drives performance in emerging and large organi-

zations, and provides strategic insights for executives and practitioners responsible for growth and business

results.

At the core of our offerings is a Membership Program that combines research, benchmarking and unlimited ac-

cess to data and analysts. The Membership Program offers insights and best practices to enable executives and

practitioners to make the right decisions about people, processes, and systems, coalesced with analyst advisory

services which aim to put the research into action in a way that is practical and efficient.

research brief - · pdf file©2015 brandon hall group. licensed for distribution by...

Documents