bmw manufacturing corp. return on investment impact study

BMW Manufacturing Corp.Return on Investment Impact Study

Gallery Tourv Review the posters with questions that will be

addressed at each evaluation levelv Complete a Post-It note with additional questions

you want addressed today

Founders of GAINS ProfileDr. Ivan Misner & Robert Davis

Purpose: To strengthen both new and established relationships

Five Elements of the GAINS Profile

1. Goals2. Accomplishments (Assets)3. Interests (Passions)4. Networks (IRS)5. Skills

Results

Let’s Network using the GAINS ProfilePresenter: Ylonda Glover, MS

Tri-Success Management Institute

GAINS ProfileFocus on I for Interests

§People are the Drivers

§Understanding People - Improves ROI

§What are your personal and professional interests?

§What are you most passionate about or what do you like doing?

Toni HodgesTH & Company

-Independent consultant, providing consulting services and workshops to private and public organizations

-Formerly managed Measurement and Evaluation for Verizon Communications and Bell Atlantic

-Bell Atlantic selected APQC Best Practice for “Measuring the Impact of Learning”

-Selected “Training’s New Guard”

-ROI Network Practitioner of the Year

-Author “Linking Learning and Performance”

Michelle M. WentzAssistant ManagerBMW Manufacturing

•Assistant Manager of Training and Associate Development at BMW Manufacturing

Ø 5000-associate company manufacturing the Z4 Roadster and X5 Sport Activity VehicleØMain responsibilities: Learning Technologies, e-Learning, Computer Applications Training, Training Measurement

•12 years of broad-based training and development experience working in the manufacturing and financial services industries

•Master of Science degree in Adult Education from the University of Wisconsin

•Bachelor of Business Administration degree in Management, also from the University of Wisconsin

Overview1. ROI Methodology

2. Program Background

3. BMW Process

4. Results

5. Conclusions

6. Recommendations

7. Lessons Learned

Collect Data After Solution

Implementation

Collect Data During Solution Implementation

Develop Evaluation Plans and

Baseline Data

Develop Objectives of

Solution

DATA COLLECTIONEVALUATION PLANNING

1. Satisfaction /Reaction

3. Application/Implementation

2. Learning 4. Business Impact

ROI Process™Methodology

Provides 6 types of data

Isolate the Effects of the

Solution

Convert Data to Monetary

Values

Calculate the Return on Investment

Tabulate Costs

Identify Intangible Measures

GenerateImpactStudy

REPORTINGDATA ANALYSIS

5. ROI

6. Intangible Benefits

Provides 6 types of data

ROI Process™Methodology

Benefits / Project BenefitsCosts =Ratio Project Costs

Net Project Benefits

ROI =Project Costs

ROI Process™Calculating the BCR and ROI

Leadership Mastery ProgramBackground

-To continue leaders’ professional development after the BMW Life Program

-Emphasis on business competencies, communication, DISC style adapting and conflict resolution skills

-3 classroom days followed by 8 hours of individualized coaching

-Target Associates: Managers, Supervisors, and Individual Contributors with Project Responsibilities

1. Two focus groups convened to determine potential business objectives (21 LMP participants)

2. Reaction data collected from BMW evaluation form (158 LMP participants)

3. Job Application data collected with 360 Degree Assessment (pre and post data collected from 16 out of 34 associates)

4. Business Impact data collected from Leadership Mastery Program Review survey (128 LMP participants responded)

5. Tangible Benefits collected from Leadership Mastery ROI Survey and actual program costs (24 out of 36 LMP participants responded)

6. All data compiled and analyzed by Return On Training (ROT) Analysis Team (3)

7. Logic Map Task Force convened for input into Conclusions/ Recommendations (2 coaches, 4 associates)

Methodology for BMWProcess Steps

-Reaction data high – ratings between 3.8 to 5.0

-Participants saw improvement in 13 of 18 competencies, 4 significantly so:

1. Communicates with Impact2. Fresh Thinking3. Positive Disposition 4. Decision Making

-Training and Coaching perceived to have a positive impact on meeting business and performance objectives

-Those that received more coaching perceived higher impact to business/performance objectives (Impact increased more than threefold for those who had more than 5 coaching sessions)

Results Reaction and Job Application

1. Select the business benefits the Leadership Mastery program most positively impacted.

2. What is the monetary value of each unit of improvement?

3. What was the change in this unit over a 30-day period?

4. To what extent do you credit the Leadership Mastery program for this change?

5. How confident are you of this estimate?

Question

1. Fewer hours spent in meetings

2. $30 per hour

3. 4 hours per month

4. 70% percent of change accredited to Leadership Mastery program

5. 80% confident

Example

ResultsCalculating Tangible Benefits

ResultsTangible Business Benefits

-14 out of 24 respondents were able to estimate tangible benefits(used in calculation)

-“Reduced number of issues reported” received highest estimated tangible benefit of $50,284

-“Fewer hours spent in meetings” was the benefit realized most often (2nd highest tangible benefit)

Participant example from Assembly Quality Department

Ø Reduced number of issues reported (Quality Improved)

Ø 10 issues reduced per month at 3 hours each @$30/hr. Ø Isolation @ 40%Ø Confidence level @ 60%Ø Annual monetary benefit = $259

ResultsBCR and ROI CalculationBenefits(Perceived Annual Monetary Value from 24 Participants -Adjusted for Isolation and Confidence)

Costs(Actual Costs for 24 Participants)

Reduced number of issues reported $50,284Reduced overtime $46,444Fewer hours spent in meetings $27,446 Reduced time to full productivity $14,275Reduced turnover $13,970Fewer associate complaints $9,714Improved customer satisfaction $3,200Reduced minutes in rework $300Improved vehicle quality measures $175

Benefit Attributable to Training: $124,008

Design and Development:($190.80 X 24 associates ) $4,579

Leadership Mastery Delivery Costs: Facilities + Instructor + Materials($1924 X 24 associates) $46,176

Associate ParticipationCosts: $26,910

$57.48/hr (FL IV) X 24 hrs X 4 associates +$53.01/hr (FL V) X 24 hrs X 9 associates +$34.02/hr (FL VI) X 24 hrs X 11 associates )(fully loaded hourly rates)

Report:$150/report X 24 associates $3,600

Coaching:$250/hour X 75 coaching hours $18,750$45.02/hour X 75 coaching hours(Participant time in coaching sessions ) $3,377

Total Training Costs: $103,392 ROI Calculation:

$124,008 (benefit attributable to training) - $103,392 (training costs ) = .199 x 100 =19.9%$103,392 (training costs)

Benefit/Cost Ratio Calculation:

$124,008 (benefit attributed to training) = 1.20 = 1.20:1$103,392 (training costs )

-Improved teamwork – 83%

-Reduced conflict – 74%

-Improved clarity of roles and responsibilities – 61%

–Improved associate job satisfaction – 39%

–Improved associate development plans – 35%

–Increased cross functionality – 30%

–Others noted: improved customer satisfaction, organizing skills, and communication

ResultsIntangible Benefits

1. Participants believed they would be able to apply the skills they learned on the job

2. Leadership competencies improved as a result of the program, according to participants: Decision-making, Communicates with Impact, Positive Disposition, and Fresh Thinking.

3. Training and coaching had a positive impact on participants’ performance

4. Training and coaching may be improved with more realistic and relevant work-related focus

5. A better understanding by participants’ management of LMP skills acquired and value expected would be beneficial

6. The LMP is leading BMW employees toward leadership with a financial return

Conclusions

RecommendationsNext Steps

1. Continue Leadership Mastery program

2. Share ROI project results with key audiences

3. Prioritize and implement LMP recommendations

4. Implement the following measurement best practices:

- Change our training design process to include the UPFRONT development of business objectives for future courses. Seek input from Senior Management so that training programs can be strategically linked to BMW’s business/ organizational objectives.

- Focus measurement efforts on learning and on-the-job application / performance impact

- Measure ROI and Business Impact for a few select training programs

Lessons LearnedPutting the Pieces Together…

•Training Design process must include link to business need, initiative, or improvement area

ØLearning objectives are not enough

ØObjective mapping process is key

ØProvides strategic link to business

ØEnsures concrete measures

•Prepare participants for accountability

ØEspecially when collecting data using participant estimates

ØTrain instructors, designers, and learners in documenting, and tracking tangible benefits

ØAction plans, business objectives, course design play a key role


•Partner with Senior Management and Financial departments

ØCommunicate ROI process (for training)

ØGet financial requirements

ØLook for ways to customize to organization in order to establish an ‘approved’ methodology


•Keep participation high

ØUtilize ‘field’ perspective whenever possible through focus groups, and task forces

ØInvolve course participants, trainers, designers, managers, etc.

ØProvides credibility, buy-in and more reliable data


•Be careful using 360º feedback for performance data (level 3)

ØIn this ROI study, used only as supporting data

Ø360º Surveying is labor intensive

ØDifficult to draw conclusions, since over time, feedback givers will change


•Anticipate personal bias

ØExpect parties with vested interest to try to influence study results

ØEvaluator needs to remain neutral and ensure objectivity

ØExplain ROI process (what’s okay and not okay to do)

ØPush back if you need to


Executive Response

Ø CFO reaction to “ROI”

Ø VP reaction to “intangibles”

Ø CEO reaction to the importance of training

Ø Overall reaction to the training organization’s link to the business

Lessons LearnedExecutive Response

Executive Response

Ø ISO Matrix Certification

Ø Auditor asked about training measurements, we offered the ROI impact study, and auditor asked for a copy of the study

Ø Received an “Honorable Mention” at Auditor’s Closing meeting as a best practice taking place within the organization!

Lessons LearnedHonorable Mention

Correlation Pyramid

-

Reaction, Learning, Performance, ROE and ROI Data

Reaction, Learning, Performance and ROE Data

Reaction, Learning and Performance Data

Reaction and Learning Data

Reaction Data

Why Are Levels of Evaluation Important?

Thank you for your attention. Let’s take a break!

Michelle M. WentzBMW ManufacturingAssistant Manager, Training and Development

[email protected]


410-956-0475

[email protected]

Rotating Table Discussions

Process

Everyone chooses a table

3 Rotations: Ø15 minute table discussion

Ø 5 minute for rotation to next table

ØBe prepared to share insights from each table discussion

Debrief

ØInsights from table discussions

ØAdditional Questions

ØSpecific challenges going forward

ØForm Networks for support and idea sharing

Michelle M. WentzBMW ManufacturingAssistant Manager, Training and Development

[email protected]


410-956-0475

[email protected]

Concluding Comments from the ASTD Board

Patrice McGuire-President Elect

Please complete the Level 1 Program Evaluation on your table!

bmw manufacturing corp. return on investment impact study

Documents