truths about training evaluation
DESCRIPTION
These slides were used during mTRANSCRIPT
![Page 1: Truths About Training Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102518/5549452db4c9050a4d8b5477/html5/thumbnails/1.jpg)
TRUTHS ABOUT TRAINING EVALUATION
David Kelly, CPLP, CRP
![Page 2: Truths About Training Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102518/5549452db4c9050a4d8b5477/html5/thumbnails/2.jpg)
A LITTLE ABOUT ME
Training Director for over 10 Years Director, Center for Learning at ACLD Certified Professional in Learning and
Performance (CPLP) Certified ROI Professional (CRP) Board Member of ASTD Long Island Member of ASTD’s National Advisors for
Chapters Member of the eLearn Magazine Editorial Board
![Page 3: Truths About Training Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102518/5549452db4c9050a4d8b5477/html5/thumbnails/3.jpg)
TODAY’S DISCUSSION
A Brief History of Measurement in Learning and Development Programs
What do Trainers Measure? Does What We Measure Even Matter?
The Question of Credibility A Better Definition for Training Success
![Page 4: Truths About Training Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102518/5549452db4c9050a4d8b5477/html5/thumbnails/4.jpg)
WHAT CAME BEFORE…
Why are we talking about ‘evaluation’ of training in the first place?
![Page 5: Truths About Training Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102518/5549452db4c9050a4d8b5477/html5/thumbnails/5.jpg)
A BRIEF HISTORY OF TRAINING EVALUATION
Don Kirkpatrick – The Four Levels of Evaluation1. Reaction2. Learning3. Behavior4. Results
Jack Phillips – ROI Methodology5. ROI
(Benefits / Costs)
![Page 6: Truths About Training Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102518/5549452db4c9050a4d8b5477/html5/thumbnails/6.jpg)
A BRIEF HISTORY OF TRAINING EVALUATION
![Page 7: Truths About Training Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102518/5549452db4c9050a4d8b5477/html5/thumbnails/7.jpg)
WHY MEASURE TRAINING?
CAPITAL INVESTMENTS
NON-CAPITAL INVESTMENTS
Buildings Equipment Tools Vehicles Companies Some Technology
Marketing Human Resources Quality Some Technology Staff Support Processes
ROI is usually measured
ROI is not typically
measured
![Page 8: Truths About Training Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102518/5549452db4c9050a4d8b5477/html5/thumbnails/8.jpg)
WHY MEASURE TRAINING?
CAPITAL INVESTMENTS
NON-CAPITAL INVESTMENTS
Buildings Equipment Tools Vehicles Companies Some Technology
Marketing Human Resources Quality Some Technology Staff Support Processes
~15% of Total Expenditures
~85% of Total Expenditures
![Page 9: Truths About Training Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102518/5549452db4c9050a4d8b5477/html5/thumbnails/9.jpg)
SATISFACTION WITH MEASURES OF SUCCESSOF LEARNING AND DEVELOPMENT
To what extent are you satisfied with the measures (value) of learning and development?
1. Very Dissatisfied – 8%2. Dissatisfied – 45%3. Satisfied – 41%4. Very Satisfied – 6%
*CEO Survey—Fortune 500 and Large Private Companies, ROI Institute
![Page 10: Truths About Training Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102518/5549452db4c9050a4d8b5477/html5/thumbnails/10.jpg)
WHAT METRICS ARE YOU USING?
![Page 11: Truths About Training Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102518/5549452db4c9050a4d8b5477/html5/thumbnails/11.jpg)
COMMON LEARNING METRICS
Cost per Employee or Hour
Course Completions
Learner Satisfaction
Attendance Test Scores
Do these metrics REALLY matter?
![Page 12: Truths About Training Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102518/5549452db4c9050a4d8b5477/html5/thumbnails/12.jpg)
MEANINGFUL PERFORMANCE METRICS Decreased Time
to Performance Reduced
Performance Cycle Times
Quality Improvements
Increased Customer Satisfaction
Increased Sales
![Page 13: Truths About Training Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102518/5549452db4c9050a4d8b5477/html5/thumbnails/13.jpg)
HERE’S MY TWO CENTS…
Measurement for measurement’s sake is a waste of time. Ask yourself… “Will I be doing anything with the data we’re collecting?”
![Page 14: Truths About Training Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102518/5549452db4c9050a4d8b5477/html5/thumbnails/14.jpg)
THE CREDIBILITY ISSUE
The truth is most training evaluation data is either irrelevant or does not hold up to
scrutiny*Bozarth-Ferguson Magic Formula used with permission
![Page 15: Truths About Training Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102518/5549452db4c9050a4d8b5477/html5/thumbnails/15.jpg)
THE CREDIBILITY ISSUE
Training: “Sales increased 40% after Training.”Marketing: “That wasn’t training, we ran advertisements during that period.”
Training: “Turnover has decreased 25% as a result of our new orientation programs.”Human Resources: “That wasn’t training, we updated hiring criteria.”
![Page 16: Truths About Training Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102518/5549452db4c9050a4d8b5477/html5/thumbnails/16.jpg)
THE CREDIBILITY ISSUE
Source: http://elearningindustry.com/subjects/free-elearning-resources/item/256-free-elearning-roi-calculators
![Page 17: Truths About Training Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102518/5549452db4c9050a4d8b5477/html5/thumbnails/17.jpg)
THE BOTTOM LINE
ROI and the metrics used for Training Evaluation are driven by companies and consultants with a vested interest in people believing these metrics matter.
In most cases, the data collected isn’t really relevant to business outcomes.
![Page 18: Truths About Training Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102518/5549452db4c9050a4d8b5477/html5/thumbnails/18.jpg)
HERE’S MY TWO CENTS…
It’s not about Learning; It’s about Performance
It’s not about what people KNOW; it’s about what people DO.
![Page 19: Truths About Training Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102518/5549452db4c9050a4d8b5477/html5/thumbnails/19.jpg)
HERE’S MY TWO CENTS…
Most executives that ask for an ROI, DO NOT WANT AN ACTUAL ROI. They want to know that the efforts had value.
![Page 20: Truths About Training Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102518/5549452db4c9050a4d8b5477/html5/thumbnails/20.jpg)
MAYBE IT’S NOT ABOUT EVALUATION…
What other ways can we determine success besides traditional training evaluation?
![Page 21: Truths About Training Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102518/5549452db4c9050a4d8b5477/html5/thumbnails/21.jpg)
The Program Stakeholders Executives Management Approving Financing of
Training Learning and Development
Professionals Program Participants Managers of Program Participants
WHO DECIDES WHAT SUCCESS MEANS?
![Page 22: Truths About Training Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102518/5549452db4c9050a4d8b5477/html5/thumbnails/22.jpg)
HERE’S MY TWO CENTS…
The time to think about how you will define success is during the DESIGN of a program, not after it’s completion.
![Page 23: Truths About Training Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102518/5549452db4c9050a4d8b5477/html5/thumbnails/23.jpg)
WHAT’S YOUR STORY?
A strong testimonial from a participant or executive on how training helped workers perform better is often more powerful than any number on a spreadsheet.
![Page 24: Truths About Training Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102518/5549452db4c9050a4d8b5477/html5/thumbnails/24.jpg)
A STRUCTURED APPROACH
![Page 25: Truths About Training Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102518/5549452db4c9050a4d8b5477/html5/thumbnails/25.jpg)
THE SUCCESS CASE METHOD
Step 1. Identify targeted business goals and impact expectations.
![Page 26: Truths About Training Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102518/5549452db4c9050a4d8b5477/html5/thumbnails/26.jpg)
THE SUCCESS CASE METHOD
Step 2. Survey a large representative sample of all participants in a program to identify high impact and low impact cases.
![Page 27: Truths About Training Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102518/5549452db4c9050a4d8b5477/html5/thumbnails/27.jpg)
THE SUCCESS CASE METHOD
Step 3. Analyze the survey data to identify: a small group of successful
participants a small group unsuccessful
participants
![Page 28: Truths About Training Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102518/5549452db4c9050a4d8b5477/html5/thumbnails/28.jpg)
THE SUCCESS CASE METHOD
Step 4. Conduct interviews with the two selected groups to: Document the nature and
business value of their application of learning
Identify the performance factors that supported learning application and obstacles that prevented it.
![Page 29: Truths About Training Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102518/5549452db4c9050a4d8b5477/html5/thumbnails/29.jpg)
THE SUCCESS CASE METHOD
Step 5. Document and disseminate the story: Report impact Applaud successes Use data to educate
managers and organization
![Page 30: Truths About Training Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102518/5549452db4c9050a4d8b5477/html5/thumbnails/30.jpg)
IT’S NOT ABOUT HOW MANY SEATS ARE FILLED
It’s not about the Training, it’s about the Performance Improvement. It’s about an ongoing process, not an event.
“Evaluating training is like evaluating the wedding instead of the marriage.”- Robert Brinkhoff
![Page 31: Truths About Training Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102518/5549452db4c9050a4d8b5477/html5/thumbnails/31.jpg)
IT’S NOT ABOUT HOW MANY SEATS ARE FILLED
Most learning and performance improvement takes place informally, and on the job.
The success of a training program is not determined via a test; it is determined by how effectively trainees use the training when they return to their work.
![Page 32: Truths About Training Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102518/5549452db4c9050a4d8b5477/html5/thumbnails/32.jpg)
SUMMARY
Data is critical to show the value and effectiveness of Learning and Performance Programs
Performance is what happens after a Learning Program, so the most important data pertains to how work has changed
![Page 33: Truths About Training Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102518/5549452db4c9050a4d8b5477/html5/thumbnails/33.jpg)
SUMMARY
Success starts with identifying desired impacts during the needs assessment
Managers of program participants are the key stakeholders for driving successful learning and performance programs
![Page 34: Truths About Training Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102518/5549452db4c9050a4d8b5477/html5/thumbnails/34.jpg)
QUESTIONS?
David Kelly E-Mail:
[email protected] Phone: (516) 474-1852 Twitter: @LnDDave Blog: http://davidkelly.me Also connect via LinkedIn or Facebook