the impact study on effectiveness of akept’s t&l training programmes
TRANSCRIPT
-
8/2/2019 THE IMPACT STUDY ON EFFECTIVENESS OF AKEPTS T&L TRAINING PROGRAMMES
1/86
1
AKEPT LEADERSHIP RESEARCH GRANT YEAR 2009
END OF PROJECT REPORT
A. Pr oject Nu mb er File at AKEPT-680-3/3/2 (16)
Pr o je ct l e ad er A ssct . Pr o f. Dr . HAN IPA H HU SSIN
Pro ject T i t le THE IM PACT STUDY ON EFFECTIVENESS OF AKEPTS T& L TRAINING
PROGRAM M ES:
Topics : KNOW LEDGE APPLICATION AN D INFLUENCING CHA NGES AM ONG
PEERS AND COLLEGES
B. Su m m a ry f o r A KEPT Re p or t
(For pub l icat ion in AKEPT Annual Report , p lease summ aries the pr oject object ives, signi f icant
resul ts achieved, research approach and t eam str ucture)
Abstract
Purpose Th is st udy base d on sho r t t er m resear ch , gr an t by Ak ad em i Kep im p in an Pen ga j ian
Tinggi M alaysia (AKEPT). The purp ose of th is stu dy is to reveal th e im pact of AKEPT Train ingProgram on lectur ers tow ard th eir contr ibu t ion on t eaching and learning act iv i t ies as peers and
col leagues contr ibut ion on kn ow ledge dissem ination pr ocesses amo ng peers and col leagues at
facul ty level .
Design/ m et hodo lo gy / ap proach De sign / m et hodo lo gy / ap p roach Da t a w ere co ll ect ed f rom
519 academ ics from all IPTA, IPTS, in M alaysia dur ing AKEPT Teaching and Learning t rainin g
w i th in 2008 to 2009
Findings Th e st udy found on ly t h ree out o f t en el em en t s o f know le dge d issem in at io n
processes show a signi f icant level ( m ore f o l low -up after tr a in ing p= 0.037, supervisor in volve
m e a f te r t ra in ing p= 0 .27 and m ore conf ident a f te r t ra in ing p= 0 .037). W here another f ind ingshow t hat anot her seven element s of know ledge dissem ination pro cesses somew hat have less
change (asking qu estio ns, suggest id es in T& L, involve pee rs, makin g a decisions, hold grou p
m eet ing , tak ing m ore t im e to t ransform p lanned into im p lementa t ion and takes t im e to
re f lect ) . A l l e lement a re modera te ly p ract iced by lecturer a t M oHE
-
8/2/2019 THE IMPACT STUDY ON EFFECTIVENESS OF AKEPTS T&L TRAINING PROGRAMMES
2/86
2
Pract ical implicat ions As co nclusio n , t h is pap er rai ses aw ar en ess an d p rovid es in it ia l
guidel ines to the AKEPTs as tra in ing centre for a l l lecturers to improve knowledge
dissem ination str ategies in t heir local universi ty.
Knowledge- In suggest ion , in t ensive organ iza t ions in fo rm ula t ing st ra teg ies on how to proper l
implement and manage their pedagogical knowledge dissemination processes are open to beexplore.
Origina l i ty / va lue Th is st udy has ext ended in ped agogica l know le dge d issem in at io n fo r it i
prob ably the f i rst to provide a com parative analysis betw een AKEPTs Train ing Centr e and local
teaching and learning tra in ing centre. I t fur ther opens up new l ines of future research
possibil i t ies.
Keywords : Know ledge Dissem inat ion Processes, Pedagogical Know ledge, Perceive
Com pet ence, Train ing
Project Object ives:
Pertain ing t o t he tr ansform ation p lan 2007 and AKEPT Centr e for Teaching and Learning,
evaluation and account abi l i ty, there are thr ee question s th at a lways need an answer fo r next
developm ent p lan in M OHE. The research object ives are:
To investigate t he impor tant of AKEPTs teaching and learning tr a in ing program s
To examine the imp act of AKEPT Train ing Program contr ibut ed to know ledge
appl icat ion in decision making in teaching and learning amo ng peers and col leagues
To examine the imp act of AKEPT Train ing Program contr ibut ed to know ledge
appl icat ion and dissem ination amo ng peers and col leagues
Research Questi on s
The most com m on reason fo r T&L evaluation is to deter m ine the effect iveness of a program
and w ays in w hich i t can be im proved. The research questions are:
1 . How impor tan t AKEPTs teach ing and learn ing program? (Leve l 2 -Re-act ion )
2 . Have the par t ic ipants ever app ly teach ing knowledge based they ga ined f rom AKEPTs
tr a in ing? (Level 2- Learning )
-
8/2/2019 THE IMPACT STUDY ON EFFECTIVENESS OF AKEPTS T&L TRAINING PROGRAMMES
3/86
3
3 . How a re the pa r t i ci pan ts pe rfo rm ing d i ffe rent l y i n o rde r to i n f luence changes among
th eir p eers and colleagues? (Level 3-influ encing chan ges change)(self-appraisal)
4 . W hat is the observat ion s resu l t f rom super io r s feedback on par t icipants qua l ity
teach ing per fo rmance? (Leve l 4- the resu l t on super io r s feedback on t he i r
subordinat es perfo rm ance) know ledge based. (evaluator)
-
8/2/2019 THE IMPACT STUDY ON EFFECTIVENESS OF AKEPTS T&L TRAINING PROGRAMMES
4/86
4
3.0 M ETHODOLOGY
3.1.Research App roach:
Quantit at ive based on quest io nnaires
1. Sect ion A: Demo graphic Prof i le and React ion t o AKEPT Tra in ing
2. Sectio n B: HOW DO YOU PERCIEVE YOUR OW NS TEACHING PERFORM ANCE
Part 1 DECISION M AKING IN TEACHING AN D
LEARNING
Part 2 KNOW LEDGE APPLICATION AN D
DISSEM INA TION
3. Sect ion C : HOW DO YOU OBSERVE YOUR SUBORDINATES TEACHING
PERFORM AN CE ON:
CONTENT AND CURRICULUM : Cont ent k now ledge of subject m at ter
KNOW LEDGE OF STUDENTS AND THEIR LEARNING: The int ellect ua l, social , phy sical, an d
personal developm ent of a l l student s.
LEARNING ENVIRONM ENTS: Learnin g envir onm ent s creat ion t hat encourag es positi ve
social interact ion, act ive engagement in learning, and self-mot ivat ion.
ASSESSM ENT: Com prehen sion and usage of a rang e of fo rm al and info rm al assessm ent
str at egies to evaluat e and ensure the cont inu ous developm ent of a l l learners.
PLANN ING AND IN STRUCTION: Inst ructio nal experiences design and creatio n ba sed on
ow n know ledge of cont ent and curriculum, stud ents, learning environm ents, andassessment.
PROFESSIONALISM : Recognit ion, par t icipat ion in, an d cont ribu ti on in tea ching as a
profession
-
8/2/2019 THE IMPACT STUDY ON EFFECTIVENESS OF AKEPTS T&L TRAINING PROGRAMMES
5/86
5
THEORITICAL FRAM EW ORKS AN D M ODEL FOR M ETHODO LOGY
3. M ETHODO LOGY FOR EVA LUATING TRAIN ING IN HIGHER EDUCATION
INTRODUCTION
This par t o f s tudy d isclose the m ethodo logy used in order to expend data f r om the des ign o f
Donald Kirkpatr ick 1959 and James Kirkpatr ick 1980 abou t m easur ing and evaluate t ra in ing.
Thus th is study focused on know ledge o f how to use the four leve ls o f eva luat ion , how to
create eva luat ion t oo ls and how to use the eva luat ion pr ocess to imp rove t ra in ing
effectiveness.
WHY EVALUATETRAINING FOR AKEPT?
How m any t im es have you heard peer s and col leagues say, I t w as a great t ra in ing program
sponsored by AKEPT but .. . ? Unfo rtu nately, th at stat emen t (or var iat ion ) is spoken far to o
of ten and re f lects a grow ing concern by both , Dean or Head o f the Depar tm ent and sen ior
m anagement t ha t t ra in ing is cost ly and not a lways w or th t he investm ent o f t ime and m oney.
This att i tud e is a lso ref lected in th e act ions of senior m anagers w ho cut tra in ing budget s f i rst
w hen t im es are tough in facu l ty .
PURPOSES OF EVALUATION
This stud y also disclose som e of t he m ain reasons to evaluate AKEPT tr aining.
- To de t er m i ne w h e t he r A KEPT t r ain in g ach ie ve s it s o b je ct iv es
- To assess t h e v alu e o f AKEPT t r ain in g p ro gr am s
- To id en t if y ar eas o f t he AKEPT p ro gr am t h at n eed s im p ro vem e nt
- To id en t if y th e ap pro pr iat e au dien ce fo r fu t ur e AKEPT p ro gr am s
- To re vie w an d re in fo r ce ke y A KEPT p r ogr am p ar t s f o r p ar t icip an t s
- To se ll A KEPT p r o gr am t o m a n ag em e n t an d pa rt i ci pan t s a m o n g M a la ysi an o f Hi gh e r
Education
LINKIN G EVALUA TION TO THE NEEDS ASSESSM ENT
Clearly, need s assessm ent is crit ical to t he success of an y tr aining init iative in AKEPT. It pro vides
th e basis for p rogram d evelopm ent and establ ishes the cr i ter ia for m easur ing th e success of
the pr ogram af te r i ts com ple t ion . For eva luat ion t o have any mean ing, i t m ust be t ied t o th e
need s assessm ent p rocess. AKEPT w ith goo d strat egies alw ays em phasis th at t he evaluatio n
-
8/2/2019 THE IMPACT STUDY ON EFFECTIVENESS OF AKEPTS T&L TRAINING PROGRAMMES
6/86
6
process shou ld ref lect specif ic perfor m ance-related out com es.
W HEN TO EVALUATE
Evaluation is an ongoing process, not just som eth ing that h appens at th e end of a session o r a
program . Consider evaluation d ur ing the session, at t he end, and after t he part ic ipants retu rn
to t he j ob .
Dur ing th e Session . As people p art icipate is skil l practice, case stud ies, exercises, sim ulat ions,
and o t her act iv it ies, observe the degree to w h ich they have m astered the content . (Leve l 1 )
At th e End o f t he Session. Part ic ipants evaluation qu estion naires w i l l indicate their personal
reactio ns to t he t raining session. (Level 1, Level 2 and Level 3)
Aft er the Training. A few weeks to several mont hs aft er th e session, observe the part ic ipants
jo b per fo rm an ce t o det erm in e w het her t hey ar e ap p ly in g w hat t hey le ar ned t o t hei r w o rk
situat ion s. (Level 4)
W HOM TO INVOLVE
Any category or num ber of peo ple can be involved in the evaluation pro cess, depending on
w hat you w ant t o know and w hat leve l eva lua t ion you conduct . The part ic ipants are d i rect ly
involved because they are the custo m ers w hose level of sat isfact ion you are try t o det erm ine.
The par t ic ipants are the m ajor source o f in fo rm at ion o f how m uch they have actua lly learned in
th e tra in ing. In addi t ion t o t he part ic ipants, involve others in t he evaluation pro cess. Survey or
interview th e part ic ipants managers, th eir co-wor kers, peers, subor dinates, or even custo m ers
or vendors. At some po in t , you need t o invo lve sen ior management .
HOW TO EVALUATE
Before developing an evaluation process, be clear of w hat yo u w ant t o evaluate. This is not as
easy as i t seem. Do you w ant t o evaluate how m uch th e part ic ipants have learned? Or do you
w ant to know w hat the par t ic ipants thought about the program and you? Do you want t o f ind
out w hether t he par t ic ipants are app ly ing w hat t hey learned on t he job? If t hey are using the
learned ski l ls and inform ation on t he job, is th eir enhanced perform ance m aking a di f feren ce to
th e organizat ion ?
FOUR-LEVEL M ODEL OF EVA LUATION
The m ost w ide ly known m ode l fo r eva lua t ing t ra in ing program s was in t roduced by Dona ld
Kirkpatr ick in 1959. It is regarded as a classic by train ing practit ione rs. Altho ugh all four levels of
m odel ( react ion, learning, behaviour, resul ts) are im port ant, you m ay choose not to evaluate at
all four levels. Stu dies sho w AKEPT , (2010) that a vast m ajorit y of o rganization s evaluat e
reaction. A signi f icantly h igh p ercentage m easur ing learning as wel l . The evaluation or
-
8/2/2019 THE IMPACT STUDY ON EFFECTIVENESS OF AKEPTS T&L TRAINING PROGRAMMES
7/86
7
m easurem ent of behaviour lags behind th e f i rst tw o levels; evaluation of resul ts f in ishes last.
Todays organizat ions are much m ore cost-conscious, and th e need t o m easure th e
effect iveness of t ra in ing w i l l continu e to grow . Should you need to un dert ake a comp rehensive
approach t o eva luat ion , you w i l l ab le to m ake appropr ia te recomm endat ions or respond
confident ly wh en someone asks you t o im prove th at tr a in ing gets resul ts.
Table 1.1 Kirkpat rick in 1959:
M easur ing Train ing Resul ts -Provides an overview o f th e fou r levels of evaluation.
W hat W ho W hen How W hy
Level
1
Reaction:
Did they l ike
it ?
Part ic ipants End o f
Program
Smile
Sheet
Determine level o f
customer
satisfact ion; m ay
ind ica te need fo rrevision.
Level
2
Learning:
Wha t
knowledge
or skil ls did
they re ta in?
Participants;
t ra iner
During,
be fo re /a f te r
program
Pre-
tes t /pos t -
te st; skil ls
appl icat ion
throu gh ro le
play s, case
studies,
exercises
Ident i fy w hether
tra iner have been
successful in d elivery
o f course content
and achieving
program ob ject ives
Level
3
Behaviour:
how a re
they
pe r fo rm ing
d i f fe rent ly?
Participants;
bosses;
subordinates;
peers
3 to 6
mon t hs af te r
program
comple t ion
Surveys;
in terv iews;
observation;
per fo rmance
appraisal
Determine extent to
w hich par t icipants
have transferred hat
they learned in the
session t o t he actual
w ork si tua t ion
Level
4
Results:
W ha t i s the
impact ont h e b o t t o m
l ine?
Participants;
contro l g roup
Af ter
comple t ion
of Level 3fo l low-up
Cost/benefi t
analysis;
tracking;opera t iona l
data
Dete rm ine w he the r
benef i ts ou tw e igh
costs; ascert aindegree of
contr ibu t ion o f
p rogram t o
organizational goals.
-
8/2/2019 THE IMPACT STUDY ON EFFECTIVENESS OF AKEPTS T&L TRAINING PROGRAMMES
8/86
8
EVALUATING TRAININ G
Level 1: Reactio n
Level 1 deals wi th part ic ipants react ion s, th at is, custo m er sat isfact ion. Level 1 evaluations
are oft en referred t o as smi le sheets, im plying that part ic ipant s react ions are based on how
m uch fun th ey had in th e tr a in ing session. For that reason, tra iners frequen tly d ismiss Level 1
evaluations as a waste of t im e.
On th e contrary, Level 1 is an imp ort ant f i rst step in determ ining the success of a
tra in ing pro gram. Part ic ipants react ion s can help you det erm ine th e effect iveness of a pro gram
and how i t can be im proved. Kirkpat r ick bel ieves th at you canno t b ypass th is f i rst level because,
as he put s it , I f they [par t ic ipants ] do not react favourab ly , they w i l l no t be m ot iva ted t o learn
(Kirkpatrick, 1994)
W hat Level 1 Cannot M easure. One o f t he prob lems w i th and t he main cause o f cr i t i c ism o f
Level 1 evaluation is th at i t is to o subject ive and oft en becom es not hing mo re than a popular i tycontest. Before constr uct ing a part ic ipants end-of-session evaluation for m , understand w hat i t
cannot and is no t in t ended to do ;
(1) it d oes not m easure learning or the abi l i ty to apply learning on t he job;
(2) i t a lso cannot m easure changes in att i tud es or b el iefs:
(3) because i t deals only w i th part ic ipant s percept ion and r eactions, a Level 1 instr um ent can
in no w ay measure organizat ional imp act;
(4) also, a l thou gh frequent ly asked, part ic ipants cannot m easure t he tra iners know ledge. Thinkabout i t . How cou ld the par t ic ipants have any way o f knowing w hat the t ra iner does and does
not know about t he sub ject? You ab i l ity t o communicateo r demons t ra te your kn ow ledge is an
ent i re ly d i f fe rent s to ry .
Decid ing What to M easure. Before you d esign a Level 1 instru m ent, you n eed to be clear about
wha t you w an t to know, w hat you w an t to know i t , and w ha t you are go ing to do w i th the
in form at ion . Do not ask fo r in fo rm at ion about somet h ing you cannot change or have no
inten tion of analyzing or report in g.
Designin g an End-of -Session Evaluat ion Form . Categories. Fi rst dec ide what you w ant to
m easure and create qu estion s or response i tem s that fa l l in to certa in categor ies, includingm any, i f no t a l l, o f the fo l low ing:
- Con t ent
- M at er ials
- In st ru ct io nal m e th od s
-
8/2/2019 THE IMPACT STUDY ON EFFECTIVENESS OF AKEPTS T&L TRAINING PROGRAMMES
9/86
9
- Trainer
- Environm en t
- Logist ics
I t i s also a good idea to pr ov ide an oppor tun i t y fo r respondents to m ake recomm endat ions
as to h ow th e program can be imp roved and also to express th eir overal l react ions to t hesession.
Format . To counteract peop le s tendency to respond t he same w ay to every i tem on a
questionn aire or survey, use a var iety of response form ats. Choose at least f our from th e
fo l lowing opt ions:
- Tw o-choice quest ions wi t h roo m for explanat ions or comm ents . These w ould include
responses such as yes o r no and agree or d isagree. Example:Did the cour se
m eet the sta ted ob ject ives? Yes No W hy or why not?
- Short answers. The item s are wr i t ten as open-ended questions and requ ire the
respondent to w r i te dow n a br ie f response instead o f just check ing a box. Example:
W hat par ts o f the w orkshop w ere most va luab le /benef icia l to you? Wh y?
- Com plete t he sent ence. W ith t h is i tem, th e respondent is asked to com ple te a
sentence. Example:What i wan t / need to know m ore abou t i s. ..
- Ratings. Part ic ipants respond to a question or statem ent using som e type of scale or
rat ing such as a Likert scale. The Likert scale m easures bot h t he dir ection (positiv e and
negative) and inten si ty (strongly posi t ive to strongly negative) of an individual s opin ion
o r a t t i tude . Example:Todays session w as enjoyab le and satisfying learning experien ce
fo r me .
1 2 3 4 5 6 7 8
- Rankings. This i tem s asks respond ents to indicate pr io r i t ies or pr eferences. Example:
Please rank each to pic in order or i t s impo rt ance or re levance to you r job: 1 = mo st
impor tan t t o 5 = least im por tan t .
- Checklist. A checklist pr ovides a laundry l ist f rom w hich part ic ipants can choose
w ords that expr ess th eir react ion s. Example:Check the w ords that descr ibe your
reaction t o t odays session:
___ __ Excee ded m y exp ect at io ns
___ __ M et m y exp ec t at io ns
___ __ Fel l sh or t o f m y ex pect at io ns
A question could also be added fo cusing on th e imp act o f th e session on t he
part ic ipant , designed to o bt ain a deeper and mo re personal response, for example: Imagine
Str ongly Disagree Strongly AgreeNeutra l
-
8/2/2019 THE IMPACT STUDY ON EFFECTIVENESS OF AKEPTS T&L TRAINING PROGRAMMES
10/86
10
th at a co-wor ker (a fr iend) of yours is th inking about atten ding th is program. He or she asks
you: Wh at w as th is p rogram l ike fo r you? How w ou ld you respond?
Evaluat ion s from Guidel ines. Evaluat ions fo rms are m ore d i f f i cu l t to construct t ha t you m ight
imagine. Use the fo l lowing guidel ines:
- Keep fo rm b r i ef . Pa r t ic ipan ts shou ld be ab le to comp lete i t qu i ck l y.
- Create a ba lance among the va r ious types o f i n fo rma t ion you are co l lect i ng. Fo r
exam ple , do not ask f ive quest ions about the inst ructor and on ly t w o about content .
- Ob ta in pa r t i ci pan ts imm ed ia te reac t ions. Have par t i ci pan ts comp le te the evaluat i on
befor e they leave the roo m. This w i l l ensure th at you r eceive a 100 percent response
rate. I t w i l l also prevent m ob m ental i ty response, the possibi l i ty of several people
gett in g together t o d iscuss th e class ei ther bef ore or w hi le they are com plet ing the
evaluation.
In terv iews. In addi t ion t o the end -of-session qu estion naires, you can use inter views to increase
th e rel iabi l i ty of th e data col lected fro m t he questionnaires. This m eth od of d ata col lect ion is
qui t e f lexib le, a l lowing the inter viewer t o prob e for m ore speci f ic answers and t o clar i fy
questions as needed. The m etho d also al lows the interview er to record spont aneous answ ers
and, therefo re, get m ore com plete picture of t he part ic ipants react ions. The int erview er can
explore in mo re detai l the reactions gleaned fro m t he questionnaires.
Plan on spend ing about th i r ty m inutes per in te rv iew. You w i l l no t be ab le to in te rv iew
every par t icipant , so select a r andom sample . I t i s im por tan t to ho ld t he in terv iews wi th in one
w eeks of t he session so th at th e exper ience is fresh in their m inds. Throu gh one-on-one
interview s, you can fur t her explore t he reasons for p art ic ipants react ion s and sol ici t
suggestion s for im provem ent . Ei ther tape t he interview s and have them t ranscr ibed, a llow ingyou t o ana lyze or in te rpre t t he responses more thorough ly , o r s imply take notes dur ing the
in terv iews.
W hen deve lop ing the in t erv iew quest ions, do not dup l ica te the qu est ions on t he
w ri t t en form . Instead, ask speci f ic questions about t he m eth ods used or the conten t covered.
For exam ple, below are several questions about th e met hods used in a tra in ing session on
leadership:
- W h a t f e el in gs d i d y o u h av e ab o u t t h e m e t h o d s u se d in t h e p r o gr am ?
- W h a t d id yo u li ke a bo u t t h e j igsa w d esi gn ?
- W h a t d id y ou l ik e ab ou t t h e le ar n in g t o u rn am e n t ?
- W h a t d id n t yo u lik e a bo u t t h e j igsa w d esi gn ?
- W h a t d id n t y o u l ik e a bo u t t h e l ea rn i ng t o u r n am e n t ?
-
8/2/2019 THE IMPACT STUDY ON EFFECTIVENESS OF AKEPTS T&L TRAINING PROGRAMMES
11/86
11
Level 2: Learnin g
Level 2 evaluation d eals w i th w hat t he part ic ipants actual ly learned d ur ing th e tra in ing session.
Kirkpatr ick defines learning as th e extent to w hich part ic ipants changes att i tudes, im prove
know ledge, and/ or increase ski l l as a resul t o f att ending the p rogram (Kirkpatr ick 1994). It is
fa r easier to d e termine w hat new know ledge or sk i ll s the p ar t icipants acqu ired th a t to f ind th e
w ays in w hich th e tr a in ing changed th eir opin io ns, values, and bel iefs.
The three m ost appropr iate m ethod s used to evaluate learning are tests, observations,
and interview s, w i th tests being th e mo st frequent . See Table 13.2 for summ ary of the
advantages and disadvant ages of each m etho d.
M et hod Advan t ages Disadvant ages
Tests:
Object ives tests
M ul t ip le cho ice
M a tch ing
True-false
Fill-in
Subject ive tests
Essay
Short answers
Easy to score
Inexpensive to use
Easy to w r i te
Inexpensive to create
Di f f i cu l t to w r i te
T im e-consuming to w r i te
Grad ing is t im e-consuming
Expensive to gr ade
Observation:
Observe beh aviour in class
Skil ls demonstrated in skil lpract ices and learning
activi ty
Imm ediate app l icat ion
Lends i tsel f to on-th e-spotcoaching and feedback
Subject ive, open to
in terpre ta t ion
Unab le to spend adequa te t ime
observing behaviour of a l l
par t ic ipants
In terv iews:
-
8/2/2019 THE IMPACT STUDY ON EFFECTIVENESS OF AKEPTS T&L TRAINING PROGRAMMES
12/86
12
Individual interviews
conducted shor t ly a f te r t he
t ra in ing
Random sam pl ing of
par t ic ipants
Can gath er m ore detai led
in fo rma t ion
Instant feedback
Time con sum ing; expensive
M ust be t igh t ly constructed t o
obt ain quanti f iable responses
Tab le 1 .2. Level 2 Evalu at io n M e t ho ds
Tests. Testing should be kept fa i r ly simple. M any tra iners give bot h a pre-test and a post- test
to get an even mo re accurate picture of wh at th e part ic ipant s have learned.
Types of test s. Fi rst de term ine w hether you w ant t o construct sub ject ive (shor t -answ er
or essay) or ob ject ive (m ult ip le-choice or true-fa lse) i tems or even a comb ination of t he tw otyp es. When constr uct ing test i tem s, consider th e t im e needed to grade the t est as w el l as the
val id i ty and th e re l iabi l ity of each i tem . M ake sure the test assesses the learning as speci f ied in
th e learning object ives. Wh en an i tem m easures wh at i t is supposed to m easure, i t has val id i ty.
Each test i tem m ust a lso be rel iable, that is, give consistent resul ts from one appl icat ion t o
anot her. For a b roader d iscussion of test constr uct ion, refer to Testing For Learning
Out com es by Deborah Grafinger Hacker, one of th e INFO-LINE booklet s publ ished by the
Am erican Society fo r Train ing and Developm ent (revised 1998). Hacker descr ibes var ious types
of t ests, provides i tem s in wr i t ing t est i tem s, d iscusses measur ing val id i ty and rel iabi l i ty of
tests, and includes a helpful and ext ensive resource l ist .
M ake sure t he test is m eaningful . Instead of asking for simp le inform ation or f actu al
recal l, ask questions that requ ire the part ic ipant s to apply or int erpret w hat t hey learned in the
session.
Quest ion fo rma ts . All m ult ip le-choice questions consist of a stem and a r esponse. The
stem pr esent s a problem , asks a question , or takes the for m o f an incomplet e stat emen t.
Respon ses include possible answers, al l of w hich m ust be plausib le. The greater t he nu m ber of
i tem s, th e bett er the t est s re l iabi l i ty. Fol low ing are som e form ats to consider:
- Correct an swer. The correct-answer fo rm at asks a sim ple question to w hich there is
only on e correct answer. I t is used pr imari ly to test t he recal l of facts. This typ e ofquest ion is appropr ia te to test p roduct knowledge, fo r exam ple .
- Best an swer. W ith t h is typ e of question, there is m ore t han one correct choice. Som e or
al l of t he choices and dr aw conclusions. This typ e of question can create m any
prob lems. Because the answ er is open to in te rpre ta t ion , the test i tem can be
chal lenged qui t e easi ly, and you m ight f ind yo urself e i ther arguing w ith t he individual
-
8/2/2019 THE IMPACT STUDY ON EFFECTIVENESS OF AKEPTS T&L TRAINING PROGRAMMES
13/86
13
or group and m ost p robab ly having to g ive cred i t fo r o ther answ ers.
- Com bined respon se. This question is th e mo st com pl icated and t im e consum ing for
both the t est w r i te r and the t est t aker . The cho ices, one or m ore tha t m ay be cor rect ,
are num bered. A second set of choices l ists com binations of possib le correct r esponses.
This t ype of que stion assesses com plex cognit ive skil ls and t he abil i t y to an alyze and
evaluate. Exercise a great deal of th ought w hen w ri t ing the i t em . Because of i ts
complex i ty , the respondent w i l l p robab ly have to spend more t ime t h ink ing about th e
i t e m .
Test-W ri t ing Guidel ines. In m ost cases, you w i l l probably choose to develop m ult ip le-choice
questions. They are easy t o grade, but not necessar i ly easy t o w r i te. To help you construct a
m ul t ip le -cho ice test th a t w i l l p rov ide va luab le in fo rm at ion about the par t ic ipants content
m astery, consider the fo l low ing guidelines:
- A vo id a ll o f t he ab ov e a n d n o ne o f t he ab ov e o p t io n s.
- M a ke su r e t h e st e m ( t h e m a in p ar t o f t h e q u est i on ) co n t ai ns m o st o f t h e i n f o r m at i o nand define t he pro blem; p lace blanks for f i l ls- ins near the en d.
- M a in ta in g ramm at i ca l consi stency o r para ll e l st ruc tu re fo r bo th the stem and the
answer choices.
- Tr y t o cr e at e ch o ic es o f eq u al l en gt h .
- Avo id amb igu i t y and read ing d if f i cu l t y by sta t i ng quest i ons i n the posi t i ve rathe r than i n
the negative.
- Ke ep t h e se n t en ce st e m s im p l e an d l im i t i t t o o n e id ea .
- Use conve rsa t i onal l anguage when ph rasing the i tem and i t s cho ices.
- A r ra nge t h e q u est i o n s i n lo gi ca l o r d er .
- D o n o t gi ve cl ue s t o t h e co r r ect a n sw e r s i n t h e qu e st i on .
To gauge re ten t ion o f the in form at ion learned in t he program, admin ister anot her test
several m ont hs fo l low ing the tr a in ing.
Observat ion. Trainers can w atch p art ic ipants pract ic ing and applying ski l ls, too ls, and
techniqu es dur ing t he session. As t he t ra iners observe part ic ipant behaviour in ski lls pract ices,
ro le plays, sim ulat ions, case studies, and ot her act iv i t ies, they can get a good idea of w hat th e
tra inees have r eal ly learned.
In terv iews. Shor t ly a f te r the t ra in ing, in te rv iew the par t ic ipants and ask them w hat t hey
learned in the session. Conduct th e interview s w ith in t he w eek fo l low ing session.
-
8/2/2019 THE IMPACT STUDY ON EFFECTIVENESS OF AKEPTS T&L TRAINING PROGRAMMES
14/86
14
Level 3: Behavio ur ( how has the t ra ining af fected the w ay par t ic ipants per fo rm on t he job? )
The cr i t ical question answ ered by Level 3 is, h ow has th e tra in ing affected th e way part ic ipant s
per form on the job?
Al tho ugh bot h m anagers and t ra in ing professionals agree that th e success of a t ra in ing
program is de termined by w hat t he par t icipants do w i th t he in format ion or sk il l s back on t he
jo b , t hese resu lt s are o f t en igno red . Le vel 3 ev aluat io n is bo t h t im e- co nsum in g a nd co st ly . I t
also requires good organizational and follow-up skil ls and processes.
Purpo se of Level 3 Evaluat ion. Use fo l low-up evalua t ion to do t he fo l lowing:
- M e asu r e l ast i n g r esu lt s o f t h e t r ai n in g
- Iden t i f y area i n wh ich t ra inees shows greatest and least imp rovemen t
- Co m p a r e f o ll ow - u p an d en d -o f -p r o gr am r e sp o n se s
Follow -up Guidel ines. Use these guidel ines for f o l low -up evaluation:
- Prepare pa r t ic ipan ts. At the end o f the t ra in ing session , te l l par t i ci pan ts that you w i l l be
conduct ing a fo l low -up evaluat ion and w hat t ype o f eva luat ion i t w i l l be .
- I f the t ra in ing wasn t e f fect i ve , f i nd ou t w hy . Encou rage pa r t ic ipan ts to i den t i f y reasons
w hy th ey haven t impro ved and wh at factors obstruct the i r p rogress. Som et imes there
are factors that inhib i t o r prevent t he appl icat ion of t he new know ledge and skil ls on
th e job. These barr iers m ight include poor environ m ental condi t ion s, lack of proper
equipm ent , the supervisor, exist ing pol ic ies and pro cedure, or even th e organizat ional
c l imate .
- Share fo l low-up evalua t ions w i th par t ic ipants managers or superv isors. These
ind iv idua ls shou ld know about p rogram r esu l ts and fo l low-up in form at ion and shou ld
be involved w ith t he part ic ipants pract ice and appl icat ion of t ra in ing.
Observat ions. Trainer o r anot her designated observer can actu al ly observe em ployees back on
th e job. Careful ly w atch emp loyees as th ey perfor m t heir rou t ine job tasks and responsibi li t ies.
To faci li tate th e process and ensure consisten cy in t he data gath er ing, create a checkl ist o f
desi red behaviours and t hen observe wheth er o r no t the em ployee is demonstra t ing these
behaviours
For exam ple, i f you are observing som eone w ho has recent ly attend ed a custom er
service program , your l ist o f custom er service behaviours might look l ike th is:
----- Sm ile
----- Gr ee t cu st o m e r w i t h Go o d m o r n in g o r G o od af t er no o n
----- U se s t h e cu st o m er s n am e .
-
8/2/2019 THE IMPACT STUDY ON EFFECTIVENESS OF AKEPTS T&L TRAINING PROGRAMMES
15/86
15
----- Asks H ow m a y I help yo u?
----- O ff er s ad d it i on al assi st a nce
----- Gi ve s t h e cu st o m e r ch o ice s.
----- St a t es w h at w e c an d o, rat h er t h an w h at w e c an t .
In terv iews. Not o n ly shou ld you in terv iew t hose wh o w ent t h rough the t ra in ing , bu t you shou ld
interview th ose w ho are affected by or closely associated w ith t he program p art ic ipants.
Possible int erviewees include th e part ic ipant s managers, co-worker s, custo m ers, or
subordinat es. The int erview qu estion s w ould have t o be careful ly constructed and designed to
focus on speci f ic appl icat ion s and behaviour changes.
Surveys. Surveys are mo re eff ic ient and less expensive than int erviews to f ind o ut w het her the
part ic ipant s are actual ly applying w hat t hey learned. Once again, do not l imi t your sources of
in fo rm at ion . Others w ho in teract w i th t hose who par t ic ipated in t he t ra in ing are o f ten are
m ore rel iable source of feedback. You w i l l w ant t o know not only i f t ra inees are using the
tra in ing on th e job but a lso how they are using i t to per fo rm bet t e r .
A cont rol grou p is a lso h elpful f or a Level 3 evaluation. To val idate you resul ts, choose a
contro l g roup o f employees f rom the sam e funct ion as the par t ic ipants who received the
tra in ing. Ask the m em ber of th e cont rol group to com plete th e same surveys, question naires,
and tests as those comple ted by the p ar t icipants . Mo n i to r the i r job per fo rm ance and compare
i t w i th t ha t o f em ployees who rece ived t ra in ing .
Regardless of w hich evaluation m eth od(s) you use, m ake sure you al low enough t im e
for t he behaviour change to t ake p lace. The length o f t ime d epends on t he program, but th ree
to six mont hs shou ld g ive the par t ic ipants am ple oppor tun i ty to app ly what t hey learned and
develop new beh aviours.
-
8/2/2019 THE IMPACT STUDY ON EFFECTIVENESS OF AKEPTS T&L TRAINING PROGRAMMES
16/86
16
Level 4: Resul t s (the im pact o f t he t ra in ing on the organ iza t ion)
Level 4 evaluation det erm ines th e imp act of th e tra in ing on t he organizat ion. Ideal ly, i t show s
how th e tra in ing has cont r ibut ed t o accompl ishing organizat ion al goals and object ives
business resul ts. I f the or ganizat ion choo ses to cond uct a Level 4 evaluation , the area of
m easurem ent m ust be the same as tho se identi f ied in the needs assessment .
To measure t ra in ings im pact on the bot tom l ine , re turn t o t he data gathered dur ing
th e needs assessment . Determ ine you cr i t ical success facto rs up-fro nt . Resul ts m easured could
inc lude any o f the fo l low ing or any measurab le i tem:
- Product ion ou t put
- Sales
- Operat ing cost s
- Cu st o m er sat isf act io n
- Qu alit y st an dard s
- Safet y records
- Turnover rat e
- Absen t eeism
- Em p lo yee gr iev an ces
- Em p l oy ee s sat i sf act io n
- Bu dget var ian ces
- Prom ot ions
Level 4 evaluation is bot h di f f icult and t im e-consum ing. I t can also be qu i te cost ly. It is
di f f icul t t o m easure because of th e many var iables th at can com e into play once the
part ic ipant s leave th e session. For t hese reasons, a Level 4 evaluation is not approp r iate f or a l lt ra in ing. From a pract ical stand point , consider Level 4 for t hese programs that are near and
dear to sen ior m anagem ent and have been ident i f ied as a top pr io r i ty .
Because of the com plexi ty of Level 4 evaluation and i ts infreq uency of u se, it is
addressed here only in ver y basic term s-just enough t o give you an idea of w hat i t involves and
th e purp oses i t serves. To explore Level 4 m ore det ai l , refer t o t he l ist ings in Append ix B,
includ ing Jack Phil l ips, Jane Ho lcom b, Don ald Kirkpatr ick and Dana Gaines Robin son and Jam es
Robinson.
As a star t ing po in t , test you cur rent kno w ledge o f m easurement te rm s by match ing the
term s in th e le f t -hand co lumn be low w i th the cor rect de f in i t ions in th e r igh t -hand co lumn (see
Appendix A for t he correct answ ers):
-
8/2/2019 THE IMPACT STUDY ON EFFECTIVENESS OF AKEPTS T&L TRAINING PROGRAMMES
17/86
17
Term Def in it ion
1 ROI a. Su rvey dat a o f em p loyee at t it udes
2 Direct Cost s b . Salar ies and benefi t s paid to p art ic ipants for th e t im e they are
at tend ing th e program
3 In dir ect Co st s c. Exp en ses f or cr eat in g t he t rain ing p ro gr am
4 Overhead d . Involves quant i ty, qual i ty, cost, t im e; easi ly convert ed to
m onetary va lue
5 Deve lopment
Costs
e. Costs of shared r esources such as heating and b uilding
main tenance
6 Hard Dat a f . Expenses of oper at ing the tra in ing departm ent t hat are notdirect ly re lated to any t ra in ing program
7 So f t Dat a g. M athemat ica l fo rmula fo r ca lcu lat ing the d i f fe rence betw een
cost and value
8 Compensation
for Part ic ipant s
h . Costs re lated t o t he del ivery of th e for t ra in ing program, such as
m ater ia ls and trainer s salary
PARTICIPANT EVALUATION AN D ACCOUNTA BILITY FOR E-LEARNIN G
Unl ike tr adi t ional evaluations, e- learning evaluations are somew hat d i f f icul t because of d el ivery
opt ions a ind iv idua l learn ing so lu t ions tha t o f ten prevent consistency and un i fo rm i ty in t he
evaluation pr ocess. For exam ple, not a l l learners w i l l com plete sel f-stu dy courses from start t o
f in ish. Som e w i l l choose only the m odu les th ey th ink t hey need. Others may have to go back
and repeat a m odule to gain a greater understand ing of i ts cont ent.
For t he m ost part , however, you can apply the same pr incip les and strategies of
tradi t ional evaluation m etho ds and levels d iscussed ear l ier in t h is chapter t o t he e- learning
process. The basic d i f ference, of course, is th at t he var ious evaluation m etho ds wi l l be do ne
electronical ly. The fo l low ing are types of evaluations that can be adapted to an e- learning
env i ronmen t .
Level 1 evaluation m easures part ic ipants react ions met hods th at can be used include
questionn aires comp leted o n-screen w ith in t he course or as e-mai l feedback. Part ic ipant
reactions can also be captur ed via onl ine focus groups or in chat roo m s.
-
8/2/2019 THE IMPACT STUDY ON EFFECTIVENESS OF AKEPTS T&L TRAINING PROGRAMMES
18/86
18
Level 2, w hich m easures w hat p art ic ipants actual ly learned, uses var ious tests. Alm ost
al l types of t ests can be adapt ed to an electron ic form at: t rue-fa lse, mult ip le-choice, essay, f i l l -
in-blanks, mat ching. Visi t W i l l iam Horto ns websi te (www.ho r ton .com ) for great examp les of
var ious test ing form ats. In addi t ion t o test ing, you can mon itor learners by observing th eir
behaviour in learning act iv i t ies such as sim ulat ions and learning games as w el l as ro le plays
conducted in chat room s.
To evaluate how w el l part ic ipants apply w hat t hey learned (Level 3), you w ould t urn t o
tradi t ional met hods such as observations of the em ployees on-t he- job perfor m ance, surveys
comple ted by t he par t ic ipants and o thers who in teract w i t h h im or her , and job per fo rm ance
records. i t m ight a lso be approp r iate to set up contr ol groups: som e emp loyees w ould
exper ience tradi t ion al classroom-based tra in ing w hi le oth ers w ould en gage in e- learning
activi t ies. Em ployees individual act ion p lans could also be m onit ored.
Level 4 evaluation fo r d istance learning is m uch th e same as i t is for tradi t ional learning
and w ould involve determ ining return on investm ent (ROI) as w el l as det erm ining benefi ts suchas a decrease in n um ber o f accidents, safety vio lat ions, tardiness, absent eeism , tur nover,
custom er com plaint s, or grievances. Business m et rics such as pro fit abil i ty (sales, revenues,
prof i t) and f inancial health (sto ck pr ice, m arket share) might a lso be targeted f or Level 4
evaluation.
As w ith any evaluation process, i t is im port ant t o col laborate wi t h th e organizat ions
leaders to d eterm ine the success cr i ter ia. W hat specif ical ly do t he key people w ant t o m easure
and how w i l l th ey use the info rm ation? In addi t ion to t he basic question s addressed in the
section o n t he fou r levels of evaluation, you m ay also be interested in evaluating th e e- learning
process by establ ishing ways to m easure th e fo l low ing:
- Ho w of ten user s lo g in?
- Ho w lo ng t hey st ay?
- H o w l o n g b e fo r e t h ey r ece iv e a r esp o nse t o a q u est i o n?
- H ow e f f ici en t t h e o n li ne sy st em i s?
ACCOUNTA BILITY FOR TRAININ G
Throughout th e business wor ld, account abi l ity for a l l funct ions is increasing. Staff fun ct ions
such as tr a in ing are expected to p rove th eir contr ibut ion and value to t he organizat ion. Topexecut ives are dem anding tha t t ra in ing depar tm ents o f fe r p roof o f the i r w or th or t ake budgets
cuts. Tw o appro aches to m easur ing the value of t ra in ing are covered below : (1) cost/ benefi t
analysis and (2) retur n on investm ent.
-
8/2/2019 THE IMPACT STUDY ON EFFECTIVENESS OF AKEPTS T&L TRAINING PROGRAMMES
19/86
19
Cost/ Benefit An alysis
The cost/ benefi t analysis looks at t he tot a l cost to pro duce a tra in ing program and at tem pt s to
quant i fy t he ben efi ts. Costs includes everything fro m th e needs assessm ent, t hrough d esign,
developm ent, del ivery, and f inal ly to fo l low -up. Bot h direct and indirect costs are used to
deter m ine the to ta l cost of t he program . As table 13.3 show s, det erm ining tra in ing costs iscomplex.
Peop le Facil i t ies M at er ials
Design and
Deve lopment
Salaries, benefit s, tr avel
for : course
developm ent , cler ical
suppor t o r consu l tan t
fee and expenses or
costs of cert i fy ing in-
house t ra iner fo r
purchased programs.
M arket ing brochures
Part ic ipants mat er ia ls
Inst ructor m anua l
Purchased resource
mater ia ls
Purchased pr ograms
Salaries, benefit s, tr avel
for : t ra iner(s) ,
part ic ipant s (average
salary), clerical suppo rt ,
consul tant f ees and
expenses
Room renta l
Equ ipm ent ren t a l
Refreshments
No tebooks
Folders
Tent cards
Paper
Penci l /pens
Fl ip chart s
Handouts
Fi lm rental /purchase
Transparencies/sl ides
Sta t ionary i t ems
Cert i f icates
Books
Art ic les repr int s
-
8/2/2019 THE IMPACT STUDY ON EFFECTIVENESS OF AKEPTS T&L TRAINING PROGRAMMES
20/86
20
Total benefi t s of the p rograms m ay be reduced costs or increased revenues direct ly
att r ibut able to th e tra in ing. In many cases, how ever, benefi t s can only be est imated .
Subtract t he to t a l costs f rom the to ta l benef i ts o f ind t he net benef i t o r the t ra in ing
program . The program is considered a f inancial success i f the costs are low er t han t he ben efi ts.
Return on Investm ent
The re turn on t he investm ent s imply shows what the payback is fo r the t ra in ing program . To
determ ine re turn on investm ent , you must w a it t h ree to s ix mont hs and even longer fo r
operat ional resul ts. The form ula for determ ining ret urn on investm ent: ROI = (net pr ogram
benefi t s/pr ogram costs) x 100. Stat ed simply, i f t ra in ing program s fa il to show a reasonable
retu rn on t he com panys investment , futur e (or even current ) tra in ing in i t iat ives are at r isk. For
m ore info rm ation on det erm ining ROI, refer to Jack Phi l l ips boo ks listed in App endix B.
SIGNIFICANCE OF THE EVA LUATION PROCESS
Evaluation is a comp lex issue. For o ne reason, m any var iables enter into the eq uation . No
m atter ho w hard you t ry to f ine- tune the eva luat ion p rocess, the rea l ity is tha t e f fect can on ly
be est im ated, and econom ic benefi ts cannot be calculated precisely. As th e ro le of t ra in ing
continu es to change and tra iners reposi t ion th emselves as perfo rm ance consul tants, there w i l l
be m ore pressure to m easure th e effect iveness of tra in ing. The good new s how ever, is th at th e
f ie ld of tra in ing and developm ent cont inues to grow at rapid pace. This trend w i l l cont inue to
provide m any oppor tu ni t ies for tra in ing professionals, both int ernal and extern al , to m ake an
impact on t he growt h and deve lopment o f ind iv iduals and organ iza t ions th roughout th e w or ld .
KEY POINTS
- Evaluat i on mus t be t i ed to the needs assessmen t p rocess.
- Eva luat ion shou ld re f lect specif ic business- re la ted or per fo rm ance- re lated outcom es.
- Ev al ua t io n in an o n go in g p r o ce ss
- Level 1 evaluat i on measu res pa r t ic ipan t react i on
- Leve l 2 evalua t ion uses tests, observat ions, and in terv iew s to m easure par t icipant
learning
- Level 3 evaluat i on measu res behaviou r , tha t i s, how the pa r t i ci pan ts app ly wha t they
learned back on t he job
- Level 4 eva lua t ion m easu res resu l t s and t ra in ing s impact on t he bo t tom l ine
- A cost / bene f it ana lysi s l ooks a t the benef i t s o f t r a in ing p rogram compared to the costto produce a t ra in ing program.
- Re tu rn on i nvestmen t add resses the degree to w h ich the cost o f a p rog ram y ie lds a
reasonab le re turn on investm ent .
The tra in ing pr ocess is a never ending cycle: analysis, design, developm ent , imp lemen tat ion,
-
8/2/2019 THE IMPACT STUDY ON EFFECTIVENESS OF AKEPTS T&L TRAINING PROGRAMMES
21/86
21
and evaluation . As m entio ned several t imes thro ughout t h is book, th e ro le of th e tra in ing
prof essional contin ues to evolve. Often th e tra iner f i l ls m any roles, including that of a
consu l tan t . W hether in te rna l o r externa l , the t ra in ing consu l tan t m ust a lw ays be m indfu l o f t he
pro cess and t he business of con sultin g.
Data Gatherin gs Str uctur es
Demo graphic Prof i le
List of Organizat ions (133 IPTA/ IPTS), 30% respond ent s from t ot al AKEPT nam e list
Figure 1 and t able 1 show s lecturer s gender of t he respondent s.
Figure 1 Lecturer s Gender
Table 1 Lecturers Gender
N %
M a le 375 42.62Female 505 57.38
43%57%
male
M ethodo logy o f Su rvey
Ex-AKEPT Candidat e(Self Evaluation)
Evaluator(Peer Evaluations)
Non-AKEPT Candidates(Self Evaluation)
Suggest Suggest
Evaluate
Evaluate
-
8/2/2019 THE IMPACT STUDY ON EFFECTIVENESS OF AKEPTS T&L TRAINING PROGRAMMES
22/86
22
Figure 2 and Table 2 show s lecturers academic age of t he respondent s.
Figure 2 Lecturers Academic Age
Table 2 Lectu rer s Academ ic Age
N %
Under 25 9 1.09
2 5 2 9 106 12.023 0 3 9 240 27.32
4 0 4 9 279 31.69
5 0 5 9 216 24.59
60 + 30 3.28
under 251.09%
2 5 - 2 9
12.02%
3 0 - 3 9
27.32%
4 0 - 4 9
31.69%
5 0 - 5 9
24.59%
60 +3.28%
-
8/2/2019 THE IMPACT STUDY ON EFFECTIVENESS OF AKEPTS T&L TRAINING PROGRAMMES
23/86
23
Figure 3 and Table 3 show s Teaching Colleagues of t he respondent s.
Figure 3 Teaching Peers and Colleagues
Table 3 Teaching Peers and Colleagues
N %
Professor 82 9.29
PM 135 15.30
Senior 375 42.62
Lecturer /Tutor 288 32.79
Prof
9.29%
PM
15.30%
Senior
42.62%
Lectu rer /
Tutor
32.79%
-
8/2/2019 THE IMPACT STUDY ON EFFECTIVENESS OF AKEPTS T&L TRAINING PROGRAMMES
24/86
24
Populat ion ( l ist of Respondent )
IPTS/IPTA
AGENSI KAUNSELING
& PENGURUSAN KREDIT
BERJAYA UNIVERSITY COLLEGE
COSMOPOINT
GLOBAL INSTITUTE OF STUDIES
HELP UNIVERSITY COLLEGE
IIUM
IMPERIA INSTITUTE OF TECHNOLOGY
IMU
INCEIF
INNOVATIVE INTERNATIONAL
COLLEGE
INSTEDTINSTITUT ADVERTISING
COMMUNICATION TRAINING SDN. BHD
INSTITUT MAKANAN
MALAYSIA
INSTITUT MUTIARA MOPAR
INSTITUT PENGURUSAN
DINAMIK (KUCHING)
INSTITUT PROFESSIONAL BAITULMAL
INSTITUT SINARAN
INSTITUT TEKNOLOGI PERAK
INSTITUTE GLOBAL
MANAGEMENT SDN. BHDINTI UNIVERSITY COLLEGE
K.K. ARAU
K.K. BANDAR DARULAMAN
K.K. BANDAR PENAWAR
K.K. BUKIT BERUANG
K.K. CHENDEROH
K.K. KEPALA BATAS
K.K. KUALA LANGAT
K.K. KUALA TERENGGANU
K.K. KUANTAN
K.K. KUCHING
K.K. KULIM
K.K. LEDANG
K.K. MENTAKAB
K.K. PAYA BESAR
K.K. ROMPIN
K.K. SEGAMAT 2
K.K. SG. PETANI
-
8/2/2019 THE IMPACT STUDY ON EFFECTIVENESS OF AKEPTS T&L TRAINING PROGRAMMES
25/86
25
K.K. TAWAU
K.K. TELUK INTAN
KBU INTERNATIONAL COLLEGE
KK
BUKIT BERUANG
KK
HULU LANGAT
KK
HULU SELANGOR
KK
SABAK BERNAM
KK BANDAR DARULAMAN
KK BAYAN BARU
KK CHENDEROH
KK HULU LANGAT
KK HULU SELANGOR
KK JELEBU
KK JEMPOLKK KUALA LANGAT
KK KUANTAN
KK LANGKAWI
KK MASJID TANAH
KK MENTAKAB
KK PASIR GUDANG
KK PAYA BESAR
KK SABAK BERNAM
KK SEGAMAT
KK SEGAMAT 2
KK SELAYANG
KK SUNGAI PETANIKK TANJUNG KARANG
KK TELUK INTAN
KK
BANDAR DARULAMAN
KK
LANGKAWI
KK
MASJID TANAH
KK
PAYA BESAR
KLIUC
KLMU
KOLEJ ANTARABANGSA
TEKNOLOGI LANJUTAN SARAWAK
KOLEJ ISLAM ANTARABANGSA
KOLEJ METROPOLITAN
KOLEJ PENGURUSAN
WARISAN
-
8/2/2019 THE IMPACT STUDY ON EFFECTIVENESS OF AKEPTS T&L TRAINING PROGRAMMES
26/86
26
KOLEJ POLY-TECH MARA
KOLEJ TEKNOLOGI TIMUR
KOLEJ TEKNOLOGI YAYASAN ALOR GAJAH
KOLEJ UNITI
KOLEJ UNIVERSITI INSANIAH
KPTM KOTA BHARU
KPTM-BANGI
KPTM-KOTA BHARU
KPTM-KUANTAN
KUIS
LIM KOK WING UNIVERSITY
MAHSA UNIVERSITY COLLEGE
MAPCU
MASTERSKILL UNIVERSITY COLLEGE OF HEALTH SCIENCES
MEDIU
NILAI UNIVERSITY COLLEGE
OPEN UNIVERSITY
POLI BALIK PULAUPOLI JB
POLI KOTA K.TRGANU
POLI MERLIMAU
POLI MUADZAM SHAH
POLI MUKAH
POLI PORT DICKSON
POLI SEBERANG PERAI
POLI SULTAN AZLAN SHAH
POLI SULTAN HJ AHMAD SHAH
POLI SULTAN MIZAN ZAINAL ABIDIN
POLI SULTAN SALAHUDDIN ABD AZIZ SHAH
POLI SULTANAH BAHIYAHPOLI TUANKU SYED SIRAJUDDIN
POLI
MUKAH
POLI
SULTAN ABDUL HALIM MUADZAM SHAH
POLI
SULTAN AZLAN SHAH
POLI
SULTAN MIZAN ZAINAL ABIDIN
POLI
SULTAN SALAHUDDIN
ABDUL AZIZ SHAH
POLI
TUANKU SYED SIRAJUDDIN
POLI. KOTA
KUALA TERENGGANU
POLI. MERLIMAU
POLI. MUADZAM SHAH
-
8/2/2019 THE IMPACT STUDY ON EFFECTIVENESS OF AKEPTS T&L TRAINING PROGRAMMES
27/86
27
POLI. PORT DICKSON
POLI. SANDAKAN SABAH
POLI. SULTAN ABDUL HALIM MU'ADZAM SHAH (POLIMAS)
POLI. SULTAN IDRIS SHAH
POLI. SULTAN SALAHUDDIN ABD AZIZ SHAH
POLI. UNGKU OMAR
POLISAS
POLITEKNIK
SULTAN SALAHUDDIN ABDUL AZIZ SHAH
RIAM INSTITUTE
OF TECHNOLOGY
SATT COLLEGE
SEGI COLLEGE SARAWAK
SEGI COLLEGE SUBANG JAYA
SELANGOR INTERNATIONAL ISLAMIC UNIVERSITY COLLEGE
SIDMA COLLEGE
SIRIM BERHAD
SUNWAY UNIVERSITY COLLEGETAFE COLLEGE
TATI UNIVERSITY COLLEGE
TWINTECH
UCSI
UDM
UIAM
UiTM
SEGAMAT
UiTM JOHOR
UITM KELANTAN
UITM MELAKA
UiTM PAHANGUITM PAHANG
UITM PERAK
UiTM PERLIS
UITM SHAH ALAM
UITM
JOHOR
UiTM
KELANTAN
UiTM
N.SEMBILAN
UiTM
P.PINANG
UiTM
PERLIS
UiTM
SABAH
UiTM
SEGAMAT
-
8/2/2019 THE IMPACT STUDY ON EFFECTIVENESS OF AKEPTS T&L TRAINING PROGRAMMES
28/86
28
UiTM
TERENGGANU
UKM
UM
UMK
UMP
UMS
UMT
UNIKL
UNIMAP
UNIMAS
UNISEL
UNITAR SARAWAK
UNITEN
UPM
UPNM
UPSI
USIMUSM
UTAR
UTeM
UTHM
UTM
UUM
-
8/2/2019 THE IMPACT STUDY ON EFFECTIVENESS OF AKEPTS T&L TRAINING PROGRAMMES
29/86
29
4.0 RESULTS
INTRODUCTION
M alaysia of Higher Education has embark seven tru sts of the Nation al Higher Education Action Plan,
(M OHE (2007), in term s of : broaden access and enhancing equi ty, Im proving t he qual i ty of teaching
and learning, enhancing research and innovatio n, str engthen ing the inst i tu t ion , intensi fying
inter nation al izat ion, encul turat ion of l i fe long learning and im proving del ivery system. From al l seven
tr usts, im proving t he qual i ty o f t eaching and learning is very crucia l in any u niversi ty in M alaysia.
Cruc ial in t e rms o f the o utcom es are necessary to be w or ld renow ned in sustainab le deve lopment
and in concentra t ion of t a len t , the abundance o f resources and the accu l tu ra t ion o f suppor t ive
governance (Salmi, 2006). According to Robin M iddlehurst (2008) she argued that 21 centu ry
context is d if fe rent f rom the past , we need create the ba lance betw een the ro le o f un iversi t ies in
th e wo r ld and integrate i t w i th the real issues happened in local . Universi t ies have pictur ed of th e
global izat ion and for t h is reason higher education w i l l need to ju st i fy i ts ro le & p art icular
contr ibu t ions.
In order t o m ake som e transform ation in t eaching and learning, M alaysia of Higher Education ,
th rough AKEPT Cent re for Teaching and Learning has been off er part ic ipants a prem ier exper ience
th rough pr ogramm es speci f ical ly designed to develop and enhance th e prof essional know ledge and
skil ls essent ial to success and excellence in higher ed ucatio n t eaching and learnin g. Objectives set by
AKEPT Cent re fo r Teaching and Learning, is first t o ensure t he staff o f local higher edu catio n
inst i tut ions are professional ly com pet ence to teach.
Second t o ensure higher education leaders are com peten t in developing inst i tut ional teaching and
learning strat egies by l inking research w ith pract ice and th ird object ives is t ensure th e academ ic
staff of local h igher education in st i tut ions are know ledgeable about and capable of st imu lat ing and
pract icing teaching and learning innovat ion t hrough bo th act ion and conceptual research.
From January 2008 up t o 2009, mo re than 8 pro fessional deve lopm ent p rogram m ers in i t ia ted by
AKEPT Centr e for Teaching and Learning and the im pact of a l l the tr a in ing programm ers cond ucted
and analyzed.
-
8/2/2019 THE IMPACT STUDY ON EFFECTIVENESS OF AKEPTS T&L TRAINING PROGRAMMES
30/86
30
4.1 SIGNIFICANT RESULTS ACHIEVED:
There are som e signif icant f ind ing on:
4.1.1-Sectio n A: Dem ograph ic Pro file and React ion t o AKEPT Training (Level 1 and 2)
4.2. 1-Secti on B: HOW DO YOU PERCIEVE YOU R OW N S TEACHING PERFORM AN CE BEFORE
AN D AFTER TRAININ G (Level 2 )
Part 1 DECISION M AKING IN TEACHING A ND LEARNING
Part 2 KNOW LEDGE APPLICATION AN D DI SSEM INA TION
4.3. 1-Secti on C : HOW DO YOU OBSERVE YOUR SUBORDIN ATES TEACHIN G
PERFORM AN CE ON (Level 3 and 4)
4.1.1 Please refer to Zuraidah (AKEPT) w orks
4.2 .1 HOW DO YOU PERCIEVE YOUR OW NS TEACHING PERFORM AN CE BEFORE AND AFTER
TRAININ G (Level 2)
Pair Sam pl e t -t est fo r it em s BEFORE and A FTER AKEPT t rai nin g for ex-AKEPT
Pair Sig. (2
ta i led)
Pai r 2 be foreAkept_33_fo l low_up_to_change - a f te rAkept_55
After AKEPTs tr a in ing course, do you do m ore fo l low-up t o th e
teaching and learning changes process in your instruct ion t o m ake
sure i t is going in t he r ight d irect ion?
0.037
Pair 4 befor eAkept_35_super ior_involve - afterAkept _57
After AKEPTs tr a in ing course, how oft en do es your super ior involve
you in t he depart men tal teaching and learning process?
0.027
Pair 8 befor eAkept_39_confiden ce_in_TL - afterAkept _61
After AKEPTs tr a in ing course, do you have confiden ce in th e t eaching
and learning decisions you m ade?
0.037
Pair 17 beforeAkept_48 - a f te rAkept_70
To wh at extent have you had access to th e necessary teaching
resources (e.g. m ater ia ls from AKEPT) to apply th e t eaching
kno w ledge aft er AKEPTs trainin g cour se?
0.008
Pair 1 8beforeAkept_49 received-af te rAkept_71
To wh at extent have you received help from AKEPT, th rough m ater ia ls
0.001
-
8/2/2019 THE IMPACT STUDY ON EFFECTIVENESS OF AKEPTS T&L TRAINING PROGRAMMES
31/86
31
from AKEPT, w i th regards to app lying the teaching know ledge aft er
AKEPTs training course?
Pair 1 9
beforeAkept_50 super io r encourage app ly a f t e rAkept_72
To what ext ent d id your super ior encourages you t o apply the ideas
and t echniques you have learned in th is AKEPTs tra in ing course afterAKEPTs training course?
0.003
Pair 2 0
befor eAkept_51 Peers and Col leagues encourage apply
a f te rAkept_73
To what ext ent d id another peer s / col leagues encourage you to apply
th e ideas and t echniq ues you h ave learned in th is AKEPTs tr aining
pro gram af te r AKEPTs training cour se?
0.034
Pair 2 1
beforeAkept_52 App ly the ideas and techn iques among peers
af te rAkept_74
To what extent d id you app ly the ideas and techn iques w i th your
peer s / colleagues after AKEPTs tr aining cour se?
0.031
4.2 .2- HOW DO YOU OBSERVE YOUR SUBORDINA TES TEACHING PERFORM AN CE (Level 3 an d 4)
Pair Sam ple t -test fo r it em s ex-AKEPT and N ON-AKEPT that has been evaluated by super ior
CONTENT AND CURRICULUM : Cont ent Know ledge of subject m att er
Pair Sig. (2
ta i led)
Pair 4 x_q13 - q13
relates cont ent t o oth er subject areas, and m akes content re levant to
student s everyday l ives.
0.024
Pair 5 x_ q1 4 - q 14
uses m ult ip le r esources including techno logies to enhance know ledge
of cont ent area(s).
0.003
Pair 6 x_q15 - q15
uses M OHE standards to create learning object ives, wr i t e lesson
plans, select appro pr iate m ater ia ls, d i rect h is/ her t eaching, and can
show evidence of how th ese standards are re lated to student
assessment.
0.025
-
8/2/2019 THE IMPACT STUDY ON EFFECTIVENESS OF AKEPTS T&L TRAINING PROGRAMMES
32/86
32
KNOW LEDGE OF STUDENTS AND THEIR LEARNING: t he i nt ellect ual, social, ph ysical, and p erso nal
deve lopment o f a l l student s.
Pair Sig. (2ta i led)
Pair 1 x_q16 - q16
clear ly and consisten tly com m unicates approp r iate and chal lenging
expectat ions that resul t in h igher achievem ent f or a l l student s
0.000
Pair 2 x_q17 - q17
consisten tly dem onstrat es det ai led know ledge of learning theor ies
and incorpora tes knowledge o f s tudent learn ing in to t he content
areas by planning instruct ional strategies th at resul t in m easurable
increase in studen t learning
0.030
Pair 3 x_q18 - q18
consisten tly provides evidence of being sensi t ive, a ler t , and
responsive t o th e int e l lectu al , socia l , physical , and personal
deve lopm ent needs o f a l l s tudents
0.023
Pair 4 x_q19 - q19
consisten tly pr ovides evidence of adaptin g instr uct ion and
assignm ent s for each individual learner based on student s
environm ents inside and out side of school
0.001
Pair 5 x_q20 - q20
consisten tly provides appropr iate evidence of effect ively adapt ing
inst ruct ion by m aking mo di f ica t ions appropr ia te t o each s tudent s
stage of developmen t
0.013
LEARNING ENVIRONM ENTS: learnin g enviro nm ent s creatio ns th at encou rage posit ive socialinter act ion, act ive engagem ent in learning, and sel f-mo tivat io n.
Pair Sig. (2
ta i led)
Pair 2 x_q22 - q22 0.005
-
8/2/2019 THE IMPACT STUDY ON EFFECTIVENESS OF AKEPTS T&L TRAINING PROGRAMMES
33/86
33
provides evidence of m anaging t im e, space, and learning resources to
prom ote act ive and equ i tab le engagement o f d iverse s tudents
Pair 3 x_q23 - q23
clear ly com m unicate c lassroom m anagem ent p lan to student s
0.008
Pair 4 x_q24 - q24
provides evidence of effect ively and consistently u sing intr insic and
extr insic mot ivat ional str ategies to engage student s in learning
0.008
Pair 5 x_q25 - q25
provides evidence of cond ucting classroom activi t ies th at incorporat e
in form at ion f rom mul t ip le cu l tu res
0.003
Pair 7 x_q27 - q27
lecturer s spoken and w ri t t en language incorporat es w el l -chosen
vocabulary that enr iches th e lesson and is approp r iate to student s
ages and int erests
0.003
ASSESSM ENT: com pr ehension and usage of a ran ge of for m al and info rm al assessm ent st rat egies to
evaluate and ensure the cont inuou s developm ent of a l l learners.
Pair Sig. (2ta i led)
Pair 1 x_q28 - q28
consisten tly and successful ly u ses di f ferent typ es of assessment to
infor m instru ct ion. The lecturer has a thorou gh understanding of
m easurem ent t heory and correct ly interp rets assessm ent r esul ts.
0.000
Pair 2 x_q29 - q29
provides evidence of col lect ing and using pre-assessm ent data t o
select or d esign clear,signi f icant , and var ied stud ent learning goals
0.000
Pair 3 x_q30 - q30
provides evidence of consistent ly choosing, developing, or u sing a
var iety o f aut hent ic and tr adi t ional classroom-based assessm ent
0.002
-
8/2/2019 THE IMPACT STUDY ON EFFECTIVENESS OF AKEPTS T&L TRAINING PROGRAMMES
34/86
34
m etho ds th at are approp r iate for m aking instruct ion al decisions.
Pair 4 x_q31 - q31
provides evidence of consistent ly involving stu dents in a var iety of
self-assessmen t act ivit ies.
0.020
Pair 5 x_q32 - q32
consisten tly develops and uses valid and equi t able grading
procedures and m akes appropr ia te adapta t ions tha t m eet t he needs
of a l l student s
0.001
Pair 6 x_q33 - q33
provides evidence of consistent ly using assessm ent data t o ef fect ively
com m unicate stu dent pr ogress to student s, parent s, and other school
personnel
0.003
Pair 7 x_q34 - q34
provides evidence o f consistent ly using resources, including
techno logy, to m aintain accurate and up-to-dat e records of stu dent
w ork, behaviors, and accom pl ishm ents to m eet individual student s
needs
0.000
Pair 8 x_q35 - q35
consisten tly u ses a var iety of assessm ents t o iden ti fy stu dent sstrengths and needs
0.003
PLANNIN G AND INSTRUCTION: instr uctio nal experien ces design and creat ion based on o w n
know ledge of content and curr iculum, stu dent s, learning environm ent s, and assessm ent .
Pair Sig. (2ta i led)
Pair 1 x_q36 - q36
clear ly explains rat ionales for instru ct ional choices
0.001
Pair 2 x_q37 - q37
lecturer s lesson plans consisten tly show evidence of int egrat ing
know ledge of subject m att er, curr iculum, stu dent s, learning th eory,
0.000
-
8/2/2019 THE IMPACT STUDY ON EFFECTIVENESS OF AKEPTS T&L TRAINING PROGRAMMES
35/86
35
and assessm ent
Pair 3 x_q38 - q38
consisten tly uses var ious instruct ional strategies to en gage and
support d iverse stu dent learners and provides mu lt ip le perspectives
on key concept s, problem s, and areas of kn ow ledge
0.000
Pair 4 x_q39 - q39
able to recognize m isconception s and m akes m odif icat ions before
and dur ing the lesson t o address student needs
0.000
Pair 5 x_q40 - q40
uses m ult ip le ro les that are appropr iate fo r instruct ion and
successfully
changes roles dur ing t he instru ct ional pro cess
0.000
Pair 6 x_q41 - q41
provides evidence of consistent ly using and mo nitor ing the
effect iveness of a var iety of
approp r iate m ater ia ls and resources to enh ance instruct ion f or
diverse learners
0.000
PROFESSIONALISM : recognit ion , part icipation in, and cont ribu tio n t o t eaching as a prof ession
Pair Sig. (2
ta i led)
Pair 2 x_q44 - q44
provides evidence of understanding obl igat ion s and respon sibi l i t ies
required by t he Higher Education Strat egic Plan 2006-2010,
prof essional or ganizat ion s eth ics docum ent s, and r elated p ol ic ies.
0.004
Pair 3 x_q45 - q45
examines personal teaching pract ices, and provides evidence of
increasing stu dent achievement as a resul t o f m odifying teaching
pract ices
0.002
Pair 5 x_q47 - q47 0.031
-
8/2/2019 THE IMPACT STUDY ON EFFECTIVENESS OF AKEPTS T&L TRAINING PROGRAMMES
36/86
36
provides evidence of part ic ipat ion in p rofessional developm ent
oppor t un i t ies based upon input f rom o thers and career goa ls
Pair 6 x_q48 - q48
provides evidence of part ic ipat ing, beyond w hat is required, in school
and system project s and m akes substant ia l cont r ibut ions in schoo l
and system project s.
0.002
-
8/2/2019 THE IMPACT STUDY ON EFFECTIVENESS OF AKEPTS T&L TRAINING PROGRAMMES
37/86
37
4.2.3 How im port ant AKEPTs teaching and learning program ? (Level 2-Learning )
Figure 4 The Im por tan ce of AKEPTs Teaching & Learnin g Pro gram
Table 4 The Im por tan ce of AKEPTs Teaching & Learning Program
Ex-AKEPT
%
Non-AKEPT
%
a) Plann ing in T & L 98.08 99.00
b) Innovat ion in M OHE 94.07 74.00
c) Exem p lary Lect urer 91.67 68.00
d) Pro fessional Train ing 96.08 93.00
e) Innovat ive in Teach ing & Train ing 96.15 99.00
f ) Case M et hod in ETC 90.39 80.00
g) SKTIP 78.02 68.00h) Teach ing & Learn ing W orkshop 94.87 87.00
i ) Six Th inking Hat 78.21 55.00
j ) Linking Research in Teach ing 89.71 80.00
0%20 %
40 %
60 %
80 %
100%a) Plannin g in T & L
b) Innovat ion in
M O H E
c) Examp lary
Lecturer
d) Profe ssional
Training
e) Innovat ive in
Teaching & Train ing
f ) Case M etho d in
ETC
g) SKTIP
h) Teaching &
Learning W orkshop
i) Six Think ing Hat
j) Lin k in g Re sear h
in Teaching
HOW IM PORTAN T TEACHING & LEARNIN G PROGRAM
Ex-AKEPT
Non-AKEPT
-
8/2/2019 THE IMPACT STUDY ON EFFECTIVENESS OF AKEPTS T&L TRAINING PROGRAMMES
38/86
38
4.2.4 How about chances and reality to att end AKEPT tr aining? (Level 2-Re-action )
Figure 5 The Impor tance of Teaching & Learning Program An d Chances to At tend
Table 5 The Im port ance of Teaching & Learning Program And Chances to At tend
Perceive Reali t y
a) Plann ing in T & L 98.08% 9.20%
b) Innovat ion in M OHE 94.07% 7.20%
c) Exem p lary Lect u rer 91.67% 4.00%
d) Pro fessional Train ing 96.08% 6.40%
e) Innovat ive in Teach ing & Train ing 96.15% 22.80%
f ) Case M et hod in ETC 90.39% 11.60%g) SKTIP 78.02% 4.00%
h) Teach ing & Learn ing W orkshop 94.87% 14.00%
i) Six Th inking Hat 78.21% 14.80%
j) Li nkin g Re sear ch in Teachin g 89.7 1% 6.0 0%
0%
20%
40%
60%
80%
100%a) Plannin g in T & L
b) Innova t ion in M OHE
c) Examp lary Lecturer
d) Professional Trainin g
e) Innovat ive in
Teaching & Train ing
f) Case M eth od in ETC
g) SKTIP
h) Teaching & Learning
Workshop
i) Six Think ing Hat
j ) Li n ki n g Re sear h in
Teaching
HOW IM PORTAN T TEACHING & LEARNIN G PROGRAM
AN D CHANCES TO ATTEND
Perceiveness
Reali ty
-
8/2/2019 THE IMPACT STUDY ON EFFECTIVENESS OF AKEPTS T&L TRAINING PROGRAMMES
39/86
39
4.2.6 How did you know AKEPT tra in ing program?
Figure 6 How AKEPTs Training Pro gram Is Know n
Table 6 How AKEPTs Training Program Is Kno w n
N Ex-AKEPT
%
N Non-AKEPT
%
Peers & Co lleagues 299 33.99 518 58.82
W W W 17 1.97 0 0.00
AKEPT Train ing Program 286 32.51 207 23.53
AKEPT W ebsit e 130 14.78 103 11.76
Ot hers 148 16.75 52 5.88
0%
10%
20%
30%
40%
50%
60%
Peers &
Collegues
W W W
AKEPT Trainin g
ProgramAKEPT W ebsit e
Others
How Did You Know AKEPT Train ing Pro gram
Ex-AKEPT
Non-AKEPT
-
8/2/2019 THE IMPACT STUDY ON EFFECTIVENESS OF AKEPTS T&L TRAINING PROGRAMMES
40/86
40
4.2.7 Do yo u kno w th e role of AKEPT (Non-AKEPT)
Figure 7 Do You Know t he Role of AKEPT?
Table 7 Do You Know th e Role of AKEPT?
N %
Yes 330 37.50%
No 550 62.50%
Yes
38%
No
63%
Do You Know t he Role of A KEPT
(Non-AKEPT)
-
8/2/2019 THE IMPACT STUDY ON EFFECTIVENESS OF AKEPTS T&L TRAINING PROGRAMMES
41/86
41
4.2.8 Have the part ic ipants ever app ly teaching know ledge based t hey gained fro m AKEPTs
tra in ing? (Level 3- Learning )
Figure 8 Teaching Technique Application
Table 8 Teaching Technique Application
Non-AKEPT
%
Ex-AKEPT
%
a) PBL 43.00 33.82
b ) Innovat ion T& L 31.00 27.94
c) e - Learn ing Techn ique 37.00 38.24
d ) M ult im ed ia In t eract ive 62.00 45.59
e) Dem onst rat ion Techn ique 42.00 45.58
f ) Case St udy 62.00 32.36
g) Cr it ic & Creat ive 36.00 25.00
h ) Exem p lary Techn ique 37.00 27.94
i) Know ledge & Dissem inat ion 55.00 33.82
0%
10 %
20 %
30 %
40 %
50 %
60 %
70 %a) PBL
b) Innovat ion T& L
c) e - Learning
Technique
d) M u l t imed ia
Interact ive
e) Demo nst rat ion
Techniquef) Case Stud y
g) Crit ic &
Creat ive
h) Examp lary
Technique
i) Know ledge &
Diss iminat ion
Have You Ever App lied Teaching Techni qu e
Non-AKEPT
Ex-AKEPT
-
8/2/2019 THE IMPACT STUDY ON EFFECTIVENESS OF AKEPTS T&L TRAINING PROGRAMMES
42/86
4 . 2 .9 How a re t he pa r t i ci pan t s p
pe ers and col leagu es? (Level 3- in
Decision M aking in Teaching an
Figure 9 Know ledge App l i cat i on
Table 9 Kn o
1) Ask ing peer s for suggest ion r
2) Fo l low -Up th e T & L changes
3) Invo lves peers for ideas
4 ) Invo l ve pee rs in depa r t m en t
5) Invo lve peers in T & L
6 ) Tendency t o pu t o f f m ak ing
7 ) Gr o u p m e e t i n g w i t h p e e r s
8 ) Con f idence w i t h ow n dec i sio
9) Us ing p lanned T & L in dec is i
10 ) D id you t ake t ime t o eva lua
4) Involve peers
depar tmenta l T
process
3) Involves peers for
ideas
2) Fol low -Up the T & L
changes to r ight
d irect ion
1) Asking peers for
suggest ion regarding
L changes
K
42
r f o rm ing d i f f e ren t l y in o rde r t o i n f l uence changes
f luencing changes change)(sel f -appraisal)
d Learni ng
and Dec ision M ak ing I n T & L
w ledge App l i ca t i on and Dec ision M ak ing In T & L
After
%
egard in g T & L ch an ges 3 3
t o r igh t d ir ect io n 4 4 .68
44.60
l T & L p ro cess 5 1
4 6
& L d ecisio n 3 6
5 3
n m ak in g 4 8
n m ak in g p ro cess 4 8
e resu lt s in T & L 4 6
0%
10 %
20 %
30 %
40 %
50 %
60 %
10) Did you take t im e to
evaluate resul ts in T & L
9) Using plan ned T & L in
decision m aking process
8) Conf idence
decision m
7) Group m eet i
peers
6) Tendency to put of f
m aking T & L decision
5) Involve peers in T & L
in
L
&
now ledge Appl icatio n and DecisionM aking In T & L
m o n g t h e ir
Before
%
40
31
45
38
33
42
29
45
45
43
i t h o w n
king
ng wi th
Af ter
Before
-
8/2/2019 THE IMPACT STUDY ON EFFECTIVENESS OF AKEPTS T&L TRAINING PROGRAMMES
43/86
43
DISCUSSION AND SUGESSION
How are the p ar t icipants per fo rm ing d i f fe rent ly in o rder t o in f luence changes am ong the i r
peer s and co lleagues? (Level 3-in fluen cing changes)(self-appraisal)
Decision M aking in Teaching and Learnin g
32/54. The resul ts show that most of the part ic ipants of the AKEPT programme responded higher
percentage of no changefor asking suggestion f rom th eir peers aft er th e program m e (33%; 40%) as
compared to before. This impl ies that academics may not feel comfortable asking or shar ing
information. The cul tural problem is in fact considered a major chal lenge in change management
in i t iat ives among the HEIs because many facul ty members consider knowledge as propr ietary and
som eth ing tha t is no t shared f ree ly (W ind & M ain , 1999). On the o th er hand, it m ay also be due to
the super io r i ty in knowledge tha t the par t ic ipants fe l t as compared to the i r peers af te r a t t end ing the
programm e w hich inhib i ts them f rom consu l t ing the i r peers.
33 / 55. M ost o f th e par t ic ipants sta ted t ha t a fo l low up w as done somew hat m ore a f te r the AKEPT
programme compared to before i t (31%; 44.68%). The resul ts imply that tra in ing provides better
direct ion to part ic ipants to check and balance their teaching and learning process.
34/ 56. The resu lts show tha t a f te r t he AKEPT program m e, most o f the par t ic ipants som ewhat d id
not consul t their peers in teaching and learning as compared to before the tra in ing (45%/ 44.6%),but th is f igure is just minimal. This could be due to the confidence the part ic ipants fe l t after the
programm e w hich may m ake them fee l super io r in know ledge as com pared to the i r peers, thus, less
consu l ta t ion . On the o ther hand, i t cou ld a lso be due to the knowledge hoard ing cu l tu re o f
academics (Wi ig , 1997) wh ich inh ib i ts them f rom know ledge shar ing .
35/ 57. Surpr isingly, m ost of the super iors remained th e same t ow ards th e part ic ipants in re lat ion t o
involving them in teaching and learning process, in fact the percentage increased for no change
(38%; 51%). This may be due to fear that super iors face in re lat ion to change in the management
th at th eir dow n l ine may br ing aft er the t ra in ing conduct ed (Tr ice and Beyer 1993).
36 / 58. M ost o f the par t icipants simi la r ly p ro jected no change, in fact m ore (33%;46%) w hen asked
how oft en t hey included col leagues and p eers in teaching and learning process. . This again could b e
due to the conf idence the par t ic ipants fe lt a f t e r the programm e which m ay make them f ee l super io r
-
8/2/2019 THE IMPACT STUDY ON EFFECTIVENESS OF AKEPTS T&L TRAINING PROGRAMMES
44/86
44
in know ledge as com pared to th eir peers, th us, less consul tat ion. On the o th er hand, i t could also be
due to the knowledge hoard ing cu l tu re o f academics (Wi ig , 1997) wh ich inh ib i ts them f rom
know ledge shar ing.
37/ 59. M ost o f t he par t ic ipants pro jected h igher no changebefor e th e AKEPT tra in ing com pared t o
after (42%;36%) for th is question. This could be due to the benefi ts that are der ived from thetr ain ing that guides th em not to procrast inate teaching or research act iv i t ies.
38 / 60. M ost o f the par t ic ipants pro jected h igher no changebefor e th e AKEPT tra in ing com pared t o
a f te r (29%;53%) fo r th is quest ion . Th is cou ld be due to the f act tha t the t ra in ing make them v isionary
leaders on t he i r own and consu l ta t ion w i th peers may be seen as w aste o f t im e w hen th ey cou ld
f igure out p rob lems by t hemselves. However , there is a negat ive connota t ion t o t h is, in t ha t i t cou ld
be viewed that some facul ty members view knowledge as a possible source of d i f ferentiat ion, and
th us defer shar ing certa in aspects of their know ledge. Unfo rtu nately, however, when kno w ledge is
viewed as a source of power i t acts as a separator between the haves and the have-nots (Wi ig,
1999) and in som e cases, kno w ledge loss occurs.
39/ 61. M ost of t he part ic ipants respond ed that t heir confidence increased after t he AKEPT
prog ramm e somew ha t more compared to be fo re i t ( 45%; 48%). Al though th i s f i gu re may be
m in imal , bu t i t i s a step t ow ards m ould ing the par t ic ipants to be a conf ident educator .
40 / 62. M ost o f t he par t ic ipants responded t ha t t hey used p lanned approach in dec ision m aking
somewhat more after the AKEPT tra in ing compared to before i t (45%; 48%). Again al l though th is
f igure may be m in imal , bu t i t i s a step t ow ards m ould ing the par t ic ipants to be m ore system at ic in
th eir teaching and learning effor ts.
41 / 63. M ost o f the par t ic ipants responded th a t they too k t im e to eva luate the i r teach ing dec isions
somew hat m ore (43%; 46%) after t he AKEPT program m e compared t o befor e i t . This im pl ies th at th e
AKEPT tra in ing tra ins them to ref lect on their teaching decisions in order to be better knowledge
disseminators.
-
8/2/2019 THE IMPACT STUDY ON EFFECTIVENESS OF AKEPTS T&L TRAINING PROGRAMMES
45/86
45
How are the part ic ipants perform ing di f ferently in order to inf luence changes amo ng their peers and
colleagues? (Level 3-inf luen cing changes change)(self-app raisal)
4.2.10 Know ledge Applicat ion and Dissemin at ion
Figure 10 Know ledge Appl icat ion and Dissem ination
Table 10 Knowledge Appl icat ion and Dissem ination
Befo re Af t er
Pe er / Co lleagu es en co ur age t o ap ply T & L 4 2% 3 9%
Superviso r encou rage t o app ly T & L gained f rom AKEPT 26% 32%
Received help f rom AKEPT on m at er ials 36% 27%
Access t o t he necessary T & L resources 36% 46%
Con t en t o f t his course accu rat ely ref lect w hat happens on your job 41% 34%
Act ually Dissem inate Teach ing Know ledge 51% 46%Act ually App ly Teach ing Know ledge 45% 45%
0%
10 %20 %
30 %
40 %
50 %
60 %
Peer / Colleagues
encourage to apply T & L
Superv isor encou rage to
apply T & L gained f rom
AKEPT
Received h elp fr om AKEPT
on m ater ials
Access to t he n ecessary T
& L resources
Cont ent o f th is course
accurate ly ref lect w hat
happens on your job
Actually Disseminate
Teaching Know ledge
Actual ly Apply Teaching
Knowledge
Know ledge Appl icat ion and Dissemin ation
Before
A f te r
-
8/2/2019 THE IMPACT STUDY ON EFFECTIVENESS OF AKEPTS T&L TRAINING PROGRAMMES
46/86
46
How are the par t ic ipants per fo rming d i f fe rent ly in o rder t o in f luence changes am ong the i r peers and
colleagues? (Level 3-inf luen cing changes change)(self-app raisal)
Know ledge Applicat ion and Dissemin at ion
42/64. Surpr isingly, most of the part ic ipants appl ied teaching knowledge before and after the
programme s imi la r ly (45%; 45%) to modera te extent . knowledge app l ica t ion happens when
knowledge is appl ied to new si tuat ions where users can learn and generate new knowledge. In the
learning process, ther e should be analysis and cr i t ical evaluation t o generat e new patt erns for fut ure
use. Decision m aking at t he organisat ion al level , innovation, and custom er re lat ionship m anagem ent
are examples of d irect knowledge appl icat ion. The appl icat ion of knowledge may generate new
knowledge or update current knowledge that wi l l have to be stored (Ear l and Scott, 1999;
M orse,2000; Par ikh, 2001). Going by th is defin i t ion used in th is study, the resul t impl ies th at th e
part ic ipant s m ay st i l l not b e adept to t ransfer their know ledge in teaching and learning process; thu s
AKEPT has to faci l itate t h is by provid ing w orkshops on ho w th is could be do ne.
43/ 65. M ost o f t he par t ic ipants d isseminated t each ing knowledge to a m odera te extent bu t be fore
the AKEPT tra in ing programme, the percentage was more as compared to after i t (51%; 46%).
Knowledge dissemination involves personal is ing knowledge and distr ibut ing i t in a useful format to
meet the speci f ic needs of users. The knowledge is art iculated in a common language using tools
that are understood by al l users. The transfer/d issemination of knowledge is vi ta l to organisat ional
success, qual i ty, and competi t iveness. In universi t ies, publ icat ions, presentat ions, websi tes, whi te
papers, teaching and learning act iv i t ies, pol ic ies, and repo rt s are examp les of m echanism used to
dissem inate/ transfer know ledge (Ear l and Scott , 1999; M orse,2000; Par ikh, 2001). In re lat ion t o th e
defin i t ion used in th is study, the part ic ipants may not real ise the importance of d isseminating their
know ledge fo r t he bet t e rment o f t he un iversi ty . Th is cou ld be due to the cu l tu re o f in f rast ructure in
the inst i tut ion which inhib i ts them from shar ing their knowledge (Shar iml lah, Chong and Hanipah,
2010). Hence, AKEPT has to conduct courses w hich guides to w ards the cul tur e of know ledge shar ing
and the inst i tu t ions wh ich the par t ic ipants be long to may need to prov ide the in f rast ructure
conducive for knowledge shar ing for example f i le shar ing soft wares. On the other hand, a smal l
num ber o f th e par t ic ipants d isseminated t each ing know ledge to a great extent a f te r th e program m e
as com pared to befor e (8%; 16%). How ever, in order f or a large percentage of part ic ipants to fo l low
th is tr ens, th e suggestion above has to be account ed for .
44/ 66. M ost of the part icipants appl ied teaching know ledge of th e AKEPT tra in ing