training effectiveness-iso prospective
DESCRIPTION
TRANSCRIPT
Training Effectiveness, design and delivery-
an ISO-9000 perspective.By Saroj Ku. Behera
1
Agenda
• Overview:• ISO Model• Requirements (9K, 14K, 18K).
• Kirpatrick Model for Training Evaluation.
• ISO-10015: Guidelines for Training.
2
Background
3
Customer
Requirements
Customer
Satisfaction
CONTINUAL IMPROVEMENT OF THEUNIFIED MANAGEMENT SYSTEM
Managementresponsibility
Resourcemanagement
Measurement,analysis,
improvement
Productrealization
ProductOutputInput
Quality Management System Model
4
6. Resource Management6. Resource Management
6.2 Human Resources
6.2.1 General6.2.1 General
Personnel whose work affect product quality to be COMPETENT on the basis of appropriate education, training, skills and experience
5
6. Resource Management 6. Resource Management (Contd.)(Contd.)
6.2 Human Resources6.2.2 Competence, Awareness and Training6.2.2 Competence, Awareness and TrainingThe organization to
• Determine necessary competence for personnel performing work affecting product quality,
• Provide training or take any other actions to satisfy these needs***,• Evaluate effectiveness of the actions taken.***
Ensure that its personnel are aware of• relevance and importance of their activities• how they contribute to the achievement of the Quality Objectives
Maintain records of education, experience, training and skills*** Changed in 2008 version
6
6. Resource Management 6. Resource Management (Contd.)(Contd.)
6.2 Human Resources (Contd)
6.2.2 Competence, Awareness and Training 6.2.2 Competence, Awareness and Training ((Guidelines)Guidelines)
Make employees at each relevant function level aware of• importance of conformance with the Quality Policy & QMS
requirements• significant impacts of their work activities on quality, actual or
potential • benefits of improved personnel performance• roles and responsibilities in achieving conformance to quality policy• the potential consequences of departure from specified procedures
7
Paragraph 6.2
Words, "affecting product quality" changed to "affecting conformity to product requirements”
Paragraph 6.2.2
Sub-clause (b) -"provide training or take other actions to satisfy these needs" changed to "where applicable training needs to be provided to achieve the necessary competence".
Sub-clause (c) – obligation to ensure that the staff attained the expected competence.
Changes in ISO-9001-2008
8
ISO-14000 & OHSAS-180014.4.2 Training Awareness &
competence
• Identify training needs• Establish training procedures
• Importance of conformance• Consequences of departure from
procedures
• Personnel competent by education, training, and/or experience
9
4.4.2 Training Awareness & competence
• Training department completes Training Needs Analysis (TNA) and training schedule, reviews at least annually and revises as necessary
• Permanent on-site contractors included in training program
• Training records maintained
10
Effectiveness
Extent to which planned activities are realized and planned results achieved.
ISO 9000:2000 3.2.14
ISO 9001 specifies requirements for a quality management system that can be used for internal application by organizations, or for certification, or for contractual purposes. It focuses on the effectiveness of the quality management system in meeting customer requirements.
ISO 9001: 2000 0.3
11
Training Design: ISO-9001
Part of Product Realization Process, Clause 7.3
Training Design Planning Design Inputs Design Output Design Review Design Verification Design Validation Control of changes
1212
7.3 Design & Development (D/D)7.3.1 Design and Development Planning7.3.1 Design and Development Planning• Plan and control Design and Development of the product• During planning, determine
• Stages of design and development• Review, verification and validation appropriate to each Design
and Development stage • Responsibilities and authorities for Design and Development
• Manage interface between different involved groups to ensure effective communication and clear responsibilities
• Update planning output, as appropriate, as Design and Development progresses
7. Product Realization 7. Product Realization (Contd.)(Contd.)
13
7.3 Design & Development (D/D) (Contd)
7.3.2 Design & Development Inputs7.3.2 Design & Development InputsDetermine the inputs related to product requirements and
maintain recordsDesign & Development inputs to include
Functional and performance requirements Applicable statutory and regulatory requirements Where applicable, Information derived from previous similar
designs Other requirements essential for Design and Development
Review inputs for adequacy. Requirements to be complete, unambiguous and non-conflicting
7. Product Realization 7. Product Realization (Contd.)(Contd.)
14
7.3 Design & Development (D/D) (Contd)
7.3.3 Design & Development Outputs7.3.3 Design & Development OutputsOutputs of D/D provided in a form that enables verification
against D/D inputs. This shall be approved prior to release D/D Outputs shall :
meet D/D input requirements provide appropriate information for purchasing, production and
for service provisions contain or reference product acceptance criteria specify product characteristics essential for safe and proper use
7. Product Realization 7. Product Realization (Contd.)(Contd.)
15
7.3 Design & Development (D/D) (Contd)
7.3.4 Design & Development Review7.3.4 Design & Development Review At suitable stages, conduct systematic D/D reviews
to evaluate the ability of results of D/D to fulfill requirements to identify any problems and proposed necessary actions
Involve and include representatives of functions concerned with design stage(s) being reviewed
Records of results of reviews and necessary actions to be maintained
7.3.5 Design & Development Verification7.3.5 Design & Development Verification Perform verification to ensure that D/D outputs have satisfied D/D
input requirements Records of results of verification & necessary actions to be
maintained
7. Product Realization 7. Product Realization (Contd.)(Contd.)
16
7.3 Design & Development (D/D) (Contd)
7.3.6 Design & Development Validation7.3.6 Design & Development Validation Perform D/D validation in accordance with planned arrangements
(see 7.3.1) to ensure Resulting product is capable of fulfilling the requirements for the
specified or known intended use or application Wherever practicable, validation to be completed prior to delivery
or implementation of the product Records of results of validation and necessary actions to be
maintained
7. Product Realization 7. Product Realization (Contd.)(Contd.)
17
7.3 Design & Development (D/D) (Contd)
7.3.7 Control of Changes7.3.7 Control of ChangesChanges in D/D to be identified and records maintained The changes to
be reviewed, verified and validated, as appropriate be approved before implementation include evaluation of the effect of changes on constituent parts
and delivered products Records of the results of validation and necessary actions
to be maintained
7. Product Realization 7. Product Realization (Contd.)(Contd.)
18
Training EvaluationTraining Evaluation
19
How the training is evaluated?
Factors affecting evaluation?
20
Evaluation- Myths
I Can’t Measure the Result of My Training Efforts.I Don’t Know What Information to Collect.If I Cannot Calculate the ROI, the Evaluation Is
Useless.My CEO Does not Require Evaluation, So, Why
Should I Do It ?There Are Too Many Variables Affecting the
Behavior Change for Me to Evaluate the Impact of Training.
Evaluation Will Lead to Criticism.I Don’t Need to Justify My Existence; I Have a
Proven Track Record.
21
FactorFactor How Factor Influences Type of Evaluation How Factor Influences Type of Evaluation DesignDesign
Change potential Can program be modified?
Importance Does ineffective training affect customer service, product development, or relationships between employees?
Scale How many trainees are involved?
Purpose of training
Is training conducted for learning, results, or both?
Organization culture
Is demonstrating results part of company norms and expectations?
Time frame When do we need the information? 22
Questions we must answer …How do participants feel about our
training program?
Are participants learning?
Is their learning transferring to the job?
Does the organization benefit from our training efforts?
23
Donald Kirpatrick’sModel
for Training Evaluation
24
Finding answers …Kirkpatrick’s 4 Levels of Evaluation
Level 1: ReactionHow do participants feel about our training
program?
Level 2: LearningAre participants learning?
Level 3: BehaviorIs their learning transferring to the job?
Level 4: ResultsDoes the organization benefit?
Source: Kirkpatrick,1998 25
DONALD KIRKPATRICK'S 4 LEVELS OF EVALUATING TRAINING
Levels Description Comments
Level 1 Reaction Trainee reaction to the course. Does the trainee like the course? Usually in the form of evaluation forms, sometimes called "smile sheets".
Most primitive and widely-used method of evaluation. It is easy, quick, and inexpensive to administer. Negative indicators could mean difficultly learning in the course.
Level 2 Learning Did trainees learn what was based on the course objectives?
Learning can be measured by pre- and post tests, either through written test or through performance tests.
Level 3 Behavior Trainee behavior changes on the job - are the learners applying what they learned?
Difficult to do. Follow-up questionnaire or observations after training class has occurred. Telephone interviews can also be conducted.
Level 4 Results Ties training to the company's bottom line.
Generally applies to training that seeks to overcome a business problem caused by lack of knowledge or skill. Examples include reductions in costs, turnover, absenteeism and grievances. May be difficult to tie directly to training.
26
Appeal of Kirkpatrick’s Model:
Assesses important areas
Widely known
Simple framework
Easy to explain and understand
27
However . . . Widely Know ≠ Widely Used
Level 1: Often (over 90%)Level 2: Sometimes (less than 35%)Level 3 & 4: Rarely (less than 15%)
Why is this a problem?Level 3 and 4 often perceived as:
Difficult to measure
Time consuming
Beyond the realm of most trainers
Level 1 result does not always mean similar Learning / Transfer / ROI results
Source: Pershing & Gilmore, 200428
Other problems …Undermines Management Partnership
Training ≠ “Silver Bullet”Training is only one strategy within entire
Performance SystemLevel 3 & 4 should include evaluations of entire
Performance System - not just training
Lacks Performance System FocusWhat about rest of Performance Environment?What factors impede / enable usage of training?
Feedback Goes to Wrong PeopleFeedback to training function only is incompleteMust include Performance Environment owners
Source: Brinkerhoff & Dressler, 200229
In Summary:An evaluation tool must be integrated
into the training programs.
Kirkpatrick’s Four Levels of Evaluation is well known, but has limitations:Unlikely completion of all 4 LevelsLacks performance system focus
30
ISO-10015Guidelines for Training
9 31
Background of ISO-10015
22 country representatives developed the draft text over several years, culminating in the publication of ISO 10015 issued December 1999.
The advantages:
• based on the process oriented concepts of the
new 9000:2000 ISO family of standards.• Offers specific guidance for training design,
review and implementation.• Provides context to organization's needs.
32
1. Linking training investment with company
performance • Key client is the organization, not only the persons being trained.
(ROI Approach)
• A company has to recognize first what is the performance challenge it faces and the causes for it.
• ISO 10015 offers a clear road map to connect training to performance goals and use it for individual and group performance improvement.
2. Organizing training on the basis of pedagogical principles and processes
• Training as a tool to cover an identified performance gap.
• Establishing an appropriate training design and effective learning processes.
• Ensures that training uses resources (finances, time and energy) optimally.
Key features of ISO 10015
33
Training: for organizational needs
34
4.1Training is 4-step processFor selecting and implementing training to
close the gaps between required and existing competence, management should monitor the following stages:
a) defining training needs;b) designing and planning training;c) providing for the training;d) evaluating the outcome of training.
35
The Training Process
36
4.2 Defining Training Needs4.2.1 General4.2.2 Defining the needs of the
organisation4.2.3 Defining the competence needs4.2.4 Reviewing Competence4.2.5 Defining Competence gaps4.2.6 Identifying solutions close to
competence gaps4.2.7 Defining the specifications for
training needs
37
Training Processes-1
38
Training Processes-2
39
Training Processes-3
40
Training Processes-4
41
4.3 Designing and planning training
4.3.1 General4.3.2 Defining the constraints4.3.3 Training methods and criteria
for their selection4.3.4 Training Plan specifications4.3.5 Selecting a training provider
42
43
4.4 Providing the training
4.4.1 General4.4.2 Providing Support
1. pre-training support2. training support3. end-of-training support
44
45
4.5Evaluating training outcomes4.5.1 General4.5.2 Preparing evaluation report
Contents of evaluation report:– specification of training needs– Evaluation criteria and description of
sources, methods and schedule for evaluation.
– Analysis of data and interpretation– Review of training costs– Conclusions and recommendations for
improvement.
46
47
5 Monitoring and improving the training process
5.1 General5.2 validation of training process
Monitoring is the key tool:– Monitoring is to be done by competent
personnel only.– Monitor records at all stages. Detect
nonconformity at all stages. Take preventive and corrective actions.
– if procedures followed and requirements met, then update the personal competence records.
– if procedures not followed yet requirements are met, revise procedures and update training records.
– If procedures followed and requirement not met, either correct the training process or seek a non-training solution.
• Identify further opportunities for improvement. 48
49
Thank You
Lawrence & Mayo Pvt Ltd
50