GIVING TRAINING A PURPOSEGIVING TRAINING A PURPOSE

Author: Harsh Vardhan

Note: Although this presentation addresses AT&T /Bell South Training (Telecom)process, this can be adapted to any training program, and is not any industry specific. The presentation addresses the need for a training program to be purposeful and assessment to be measured, against set objectives and goals.

GIVING TRAINING A PURPOSEGIVING TRAINING A PURPOSE

To train with a clear ‘Performance Objective’, facilitates planning the course, ‘Assessment Tools’ as per standard and deliver against a set time frame.

GIVING TRAINING A PURPOSEGIVING TRAINING A PURPOSE

3 COMPONENTS OF PERFORMANCE OBJECTIVE – TASK, ASSESSMENT STANDARDS AND CONDITIONS.1.The Performance Objectives have to be set as per module (Logical Sections) in addition to the Terminal Performance Objectives (At the end of the program).

2.This would clearly set up expectations as per the shorter time frames, which is the usual industrial hazard one lives with.

GIVING TRAINING A PURPOSEGIVING TRAINING A PURPOSE

3 COMPONENTS OF PERFORMANCE OBJECTIVE 3.Before the course ware is designed (or is received frequently as in

case of AT&T /Bell South Product Training), a performance Objective needs to be written for each ‘Performance Gap’, that has been identified, for which training is seen as an answer.

GIVING TRAINING A PURPOSEGIVING TRAINING A PURPOSE

3 COMPONENTS OF PERFORMANCE OBJECTIVE It is an exact description of what a Learner would demonstrate the ‘Targeted Competency’ in terms of required performance, under the predefined set of conditions.

GIVING TRAINING A PURPOSEGIVING TRAINING A PURPOSE

WHY USE PERFORMANCE OBJECTIVES

• It describes specifically the destination of the Learner – Skills/Knowledge/Attitude, she/he would have learned.

• With the Learner’s themselves participating in the course development, they would see more purpose in the training and contribute to their own learning.

GIVING TRAINING A PURPOSEGIVING TRAINING A PURPOSE

WHY USE PERFORMANCE OBJECTIVES

• Performance Objectives take vagueness out of the learning process.

• Learners know what task they are going to perform, how well to perform - before being judged competent and the assessment parameters.

GIVING TRAINING A PURPOSEGIVING TRAINING A PURPOSE

WHY USE PERFORMANCE OBJECTIVES• The trainer/ course designer is less likely to go off on a tangent. A

course map, the content and the appropriate learning tasks can be assigned.

• Based on the minimum standard required, the trainer can determine the assessment tool to assess competency.

• During the actual training, it ensures that all involved remain on track.

GIVING TRAINING A PURPOSEGIVING TRAINING A PURPOSE

COMPONENTS OF PERFORMANCE OBJECTIVETASK

This is stated as an action verb, that describes easily what the learner must be able to do; to demonstrate competency. It has to be written in terms of measurability and observable behavior.

GIVING TRAINING A PURPOSEGIVING TRAINING A PURPOSE

COMPONENTS OF PERFORMANCE OBJECTIVEASSESSMENT STANDARDS

This addresses the need to judge the level/s of participants, at the thru put. It also ascertains, how they will perform post; in order to determine the desired competency level.

E.g. – “The Performance is satisfactory when…”

GIVING TRAINING A PURPOSEGIVING TRAINING A PURPOSE

COMPONENTS OF PERFORMANCE OBJECTIVEASSESSMENT STANDARDS • It has to be written to match capabilities of beginners and not the

level of expert’s.• It has to very specific. The learner should be clear on their assessed

and deemed competency. This alone will be the focus of the teaching.

GIVING TRAINING A PURPOSEGIVING TRAINING A PURPOSE

COMPONENTS OF PERFORMANCE OBJECTIVECONDITIONS

This specifies the circumstances under which, the desired performance will be demonstrated. The statement often begins with,” Given”.

GIVING TRAINING A PURPOSEGIVING TRAINING A PURPOSE

CONDITIONS

E.g. : 1. The Team Member will be able to achieve 5 SPD (Sales Per Day)by the 10th day in Incubation (On the Job Training - OJT), given that she/he has been allowed to shadow 10 SPD performer/s for 10 hours in continuity.

2. That she has been coached by an Asst. Coach/Mentor for a total of 9 hours, post barging (listening to ‘live’ calls)

in the previous 9 days.

GIVING TRAINING A PURPOSEGIVING TRAINING A PURPOSE

CONDITIONS

E.g. :That the Lead Set has not been used (‘Virgin Lead Set’ is a rarity), for more than 2 days prior to her going on the floor and suffices for a group of 15 members for a period of 15 days.

GIVING TRAINING A PURPOSEGIVING TRAINING A PURPOSE

Conditions may include such factors, as whether the participants will be assessed – •Orally or in Writing; •Whether the learner will be allowed to use references, such as handouts, notes or any other tools; •Whether the performance will be evaluated ,during the classroom training, incubation (OJT).

GIVING TRAINING A PURPOSEGIVING TRAINING A PURPOSE

The important issue is that the assessment, should match the actual job, as closely as possible. Therefore all the necessary tools and resources should be made available.

CHECKING THE LEARNING – ASSESSMENTCHECKING THE LEARNING – ASSESSMENT

CHECKING THE LEARNING – CHECKING THE LEARNING – ASSESSMENTASSESSMENT

CHECKING THE LEARNING – ASSESSMENTCHECKING THE LEARNING – ASSESSMENT

Assessment is about collecting evidence of a participant’s performance, to ascertain whether a candidate has reached; a defined performance standard.Competence can be measured as norm-referred (in comparison with other participants of an equal level) or definition-referred (measured against clearly defined criteria).

CHECKING THE LEARNING – ASSESSMENTCHECKING THE LEARNING – ASSESSMENT

WHY ASSESS?

CHECKING THE LEARNING – ASSESSMENTCHECKING THE LEARNING – ASSESSMENT

Finding out about the Learner’s PerformanceWriting the performance objective, is the starting of the designing the

learning process. While designing it is important to keep in mind, that this has to result in a particular performance, and there has to be a measurement process; at different stages of the learning.

Once the ‘Performance Objective’ has been defined, if proper assessment is not in place,then there is no way to ascertain, if training has made a difference to skills, knowledge and attitude.

CHECKING THE LEARNING – ASSESSMENTCHECKING THE LEARNING – ASSESSMENT

ORGANIZATIONAL BENEFITS

The organization has to assess on the ROTI (Return on Training Investment). The future organizational Human Resources Development can be planned on a similar basis.

Human resources development is cyclic

in nature - as we keep assessing the

performance gap and training to ever

increasing performance levels, and

addressing constantly shifting paradigms;

transparent and unbiased assessment is

non-negotiable.

CHECKING THE LEARNING – ASSESSMENTCHECKING THE LEARNING – ASSESSMENT

BENEFIT FOR THE INDIVIDUAL

It also addresses the human need to accomplish, and measure success as result of the endeavor. The report card, balances the achievements with opportunities.

Assessment can also be used in

the feedback loop, to ensure that

the design and delivery of training;

is effective.

CHECKING THE LEARNING – ASSESSMENTCHECKING THE LEARNING – ASSESSMENT

WHERE DO I BEGIN?

CHECKING THE LEARNING – ASSESSMENTCHECKING THE LEARNING – ASSESSMENT

WHAT DO WE WANT TO ASSESS? COMPETENCE!

COMPETENCE

The quality of being competent; adequacy; possession of required skill, knowledge, qualification, or capacity, sufficiency;

COMPETENCE

A sufficient quantity, legal capacity or

qualification based on the meeting of

certain minimum requirements of age,

soundness of mind, citizenship, or the like.

CHECKING THE LEARNING – ASSESSMENTCHECKING THE LEARNING – ASSESSMENT

COMPETENCE

For the purpose of this training, we define competence as – the possession of sufficient knowledge, skills and attitudes, to allow the completion of a task, to a defined standard under specified conditions.

CHECKING THE LEARNING – ASSESSMENTCHECKING THE LEARNING – ASSESSMENT

WHAT TYPES OF ASSESSMENT’S

ARE THERE?

CHECKING THE LEARNING – ASSESSMENTCHECKING THE LEARNING – ASSESSMENT

NORM REFERENCED AND CRITERION REFERENCED ASSESSMENTMost school students are familiar with testing where the scores are ranked on a list – often as a percentage or as a grade.

They use this to compare their performance with their peer’s. This is called as ‘Norm Referenced Assessment’. We call it ‘Stack Racking’ in Operational Performance Evaluation. This type of assessment is particularly useful for –

•Classifying Learners

•Selecting learners for fixed quota requirements

•Comparing the learning achievement, between learners in a group

CHECKING THE LEARNING – ASSESSMENTCHECKING THE LEARNING – ASSESSMENT

FORMATIVE AND SUMMATIVE ASSESSMENTWe often begin assessing trainees, without knowing them. When a

trainer asks a trainee to recall a recently covered point or asks to summarize previous learning, assessment is taking place.

FORMATIVE ASSESSMENT Sometimes a test can be conducted, before continuing with the instructions. Most assessments, during training are informal. This assessment methodology,is called Formative Assessment .

CHECKING THE LEARNING – ASSESSMENTCHECKING THE LEARNING – ASSESSMENT

SUMMATIVE ASSESSMENT

This method, measures over all attainment. It is used to determine the degree of mastery of a given task and to pinpoint the part of task, not yet mastered.

Its purpose is to help the learner

focus, on the particular learning,

necessary to progress towards

the required standard of

performance.

CHECKING THE LEARNING – ASSESSMENTCHECKING THE LEARNING – ASSESSMENT

APPROPRIATE ASSESMENT FOR THE THREE DOMAINS OF LEARNING

CHECKING THE LEARNING – ASSESSMENTCHECKING THE LEARNING – ASSESSMENT

The three domains of learning –

Knowledge, Skills and Attitudes, may require different approaches, when it comes to assessment. One form of assessment may be insufficient to asses, the different domains.

CHECKING THE LEARNING – ASSESSMENTCHECKING THE LEARNING – ASSESSMENT

SKILLS • Demonstration of a skill• Observation – Using a checklist or rating scale• Witness testimony – Confirmed by somebody, other than the

assessor • Examination of the produce of the assessment – Using a

checklist or rating scale• Event Record – A record of the observations of the trainee,

‘performing the learnt skill’ by the trainee or the trainer • Project report – This is suitable for assessing applications of

learning, where a number of skills are combined . E.g. – design, choice of resources, planning, practical application

CHECKING THE LEARNING – ASSESSMENTCHECKING THE LEARNING – ASSESSMENT

KNOWLEDGE • Oral questions to the trainees

• Written answers which can be multiple choice, alternative choice, short answers, sentence completion, matching the correct answer, essay writing, calculations, statistical interpretation, recognition, drawing diagrams, etc.

CHECKING THE LEARNING – ASSESSMENTCHECKING THE LEARNING – ASSESSMENT

ATTITUDES

It is very difficult to measure attitudes and values. Most assessors’ assess just Knowledge and Skills. However there are many Psychometrics based test available. To get a fair assessment test needs to be conducted - both Pre and Post training.

CHECKING THE LEARNING – ASSESSMENTCHECKING THE LEARNING – ASSESSMENT

RELIABILITY AND VALIDITY

Ensuring that the assessment is reliable and valid, is of vital importance. Specifically, unreliable or invalid assessment may affect the trainee’s chances of success. Similarly unqualified trainees, may undertake work beyond their capabilities.

CHECKING THE LEARNING – ASSESSMENTCHECKING THE LEARNING – ASSESSMENT

RELIABILITY Reliability means that the assessment can be relied upon to give an

accurate statement, about the competence of individuals in relation, to the standards defined.

In the design stage it is important to test the assessment process, in the same manner that the car manufacturer’s put their models, through a string of rigorous road and lab tests.

They aim to ensure reliability, and assessment works in exactly the same way. If we test thoroughly, then we have a much greater chance of success.

CHECKING THE LEARNING – ASSESSMENTCHECKING THE LEARNING – ASSESSMENT

RELIABILITY One of the key issues, that need to be addressed in determining

reliability, is the number of performances, which will be required to demonstrate competency.

Will the learner be deemed capable, if s\he performs the task just once, or if not, how many times will be required to prove competency?

There are number of statistical techniques which are used to test reliability, once the trainees ‘go live’. However, it is often simpler to undertake a review of assessment to check reliability.

CHECKING THE LEARNING – ASSESSMENTCHECKING THE LEARNING – ASSESSMENT

VALIDITYValidity is the extent to which an assessment measures the specific

skills, knowledge and attitudes, it is supposed to measure.

There are 4 aspects of validity which must be considered:• FACE• CONTENT• CONSTRUCT• PREDICTIVE

CHECKING THE LEARNING – ASSESSMENTCHECKING THE LEARNING – ASSESSMENT

FACE VALIDITY

Trainees being assessed have a right to be assessed on the areas that they have prepared, or been trained upon. They also have a right to know, that they are being assessed.

CHECKING THE LEARNING – ASSESSMENTCHECKING THE LEARNING – ASSESSMENT

CONTENT VALIDITY

It is an act, which helps define, that the assessment actually tests the content, which was defined in the performance objective.

CHECKING THE LEARNING – ASSESSMENTCHECKING THE LEARNING – ASSESSMENT

CONSTRUCT VALIDITY An assessment will have construct validity, if those taking the test

respond in a way, in which the assessor intended. Sometimes, those being tested complain, that they weren’t able to understand the question or directions, which they were given.

CHECKING THE LEARNING – ASSESSMENTCHECKING THE LEARNING – ASSESSMENT

PREDICTIVE VALIDITY In designing assessment, we want to predict whether a trainee, will be able to demonstrate the competency at other times, than those during the assessment. Not doing so, can lead to serious consequences.

CHECKING THE LEARNING – ASSESSMENTCHECKING THE LEARNING – ASSESSMENT

COMPLEX TASKS

Assessors need to be aware that if successful performance depends on a combination of skills, knowledge and attitudes, then the assessment process must take account .

CHECKING THE LEARNING – ASSESSMENTCHECKING THE LEARNING – ASSESSMENT

THINGS TO WATCH OUT FOR IN ASSESSMENT

CHECKING THE LEARNING – ASSESSMENTCHECKING THE LEARNING – ASSESSMENT

THE HALO EFFECT

The Halo Effect occurs when an assessor rates a learner’s achievement based on prior performance.

We have to be careful, that we

are not influenced by factors,

other than the performance of

the participant, at the time of the

assessment and the behavior

being assessed.

CHECKING THE LEARNING – ASSESSMENTCHECKING THE LEARNING – ASSESSMENT

HAWTHORNE EFFECT

The Hawthorne effect occurs, when people act differently under assessment conditions, than they would normally. To avoid this effect, assessment should be carried out under normal conditions

CHECKING THE LEARNING – ASSESSMENTCHECKING THE LEARNING – ASSESSMENT

StereotypingIf we judge people, on the basis of our perception of the group, to which they belong, we are stereotyping. Groups with which we associate could range on basis of age, gender, culture, occupation, language and others. It is difficult, but necessary, to remain objective in our assessment.

CHECKING THE LEARNING – ASSESSMENTCHECKING THE LEARNING – ASSESSMENT

Error of LeniencyAssessors have their own standard, and tend to make appraisals against these. An assessor may be positively lenient, that s\he rates people higher than they have performed.

Alternatively, an assessor could be negatively lenient, giving people lower assessments than they deserve.

Careful design of assessment forms and training of assessors can help to avoid this type of assessment error.

CHECKING THE LEARNING – ASSESSMENTCHECKING THE LEARNING – ASSESSMENT

Plan well and get started. Thank you.