application of ari skill retention maintenance tasks · research report 1538 application of ari...
TRANSCRIPT
00C U.S. Army Research Institutefor the Behavioral and Social Sciences
N• Research Report 1538
Application of ARI Skill RetentionModel to Wheel Vehicle
Maintenance Tasks
Douglas Macpherson, Chavis Patterson, and Angelo MirabellaU.S. Army Research Institute
DTICELECTEMAR 15 19901
SEAugust 1989
Approved for public release; distribution is unlimited,
U.S. ARMY RESEARCH INSTITUTE
FOR THE BEHAVIORAL AND SOCIAL SCIENCES
A Field Operating Agency Under the Jurisdictionof the Deputy Chief of Staff for Personnel
EDGAR M. JOHNSON JON W. BLADESTechnical Director COL, IN
Commanding
Technical review by
Joseph D. Hagman Aoeession ForJohn J. Kessler Nzes Giif j
ITIC TABUnanneuneod .Justftleation
DyDistribut ion/Availability Codes
Dist special
L Avail &ad/or
--
NOTICES
DISTX BUTION: Primary distribution of this re'Prot has been made by ART. Please addrcsscorresp ndence concerning distribution of reports to&L[.S. Army Research Institbte for theBehavioratbq Social Sciences, ATTN: PERI-POX, 500tEisenhower Ave., Alexandria,Virginia 22333-30-
FINAL DISPOSITION: This report may be destroyed when it is no longer needed. Please donot return it to the U.S. Army Research Institute for the Behavioral and Social Sciences.
NOTE: The findings in this report are not to be construed as an official Department of theArmy position, unless so designated by other authorized documents.
UNCLASSIFIEDSECURITY CLASSIFICATION OF THIS PAGE
REPOT DCUMNTATON AGEForm ApprovedREPOR DOCMENTAION AGE0MB No. 0704-0Ie88
la. REPORT SECURITY CLASSIFICATION lb. RESTRICTIVE MARKINGSUnclassified --
2a. SECURITY CLASSIFICATION AUTHORITY 3. DISTRIBUTION/AVAILABILITY OF REPORT-- __Approved for public release;
2b. DECLASSIFICATION/DOWNGRADING SCHEDULE distribu.tion is unlimited.
4. PERFORMING ORGANIZATION REPORT NUMBER(S) 5. MONiTOR:NG ORGANIZATION REPORT NUMBER(S)
ARI Research Report 1538 1
6a. NAME OF PERFORMING ORGANIZATION |6b. OFFICE SYMBOL 7a. NAME OF MONITORING ORGANIZATION
U.S. Army Research Institute (If applicable)
SPERI-II6c. ADDRESS (City, State, and ZIP Code) 7b. ADDRESS (City, State, and ZIP Code)
5001 Eisenhower AvenueAlexandria, VA 22333-5600
Ba. NAME OF FUNDING/SPONSORING 8b. OFFICE SYMBOL 9. PROCUREMENT INSTRUMENT IDENTIFICATION NUMBERORGANIZATIONU S. Army Research (If applicable)Insti-,te for the Behavioral I!-
and Social Sciences PERI-I8c. ADDRESS (City, State, and ZIP Code) 10. SOURCE OF FUNDING NUMBERS
PROGRAM PROJECT TASK WORK UNIT5001 Eisenhower Avenue ELEMENT NO. NO. NO. ACCESSION NO.Alexandria, VA 22333-5600 63744A 795 344 140
11. TITLE (Include Security Classification)
Application of ARI Skill Retention Model to Wheel Vehicle Maintenance Tasks
12. PERSONAL AUTHOR(S)Douglas Macpherson, Chavis Patterson, and Angelo Mirabella
13a. TYPE OF REPORT 113b ' TIME COVERED 114. DATE OF REPORT (Year, Month, Day) 115. PAGE COUNTInterim FROM_87/09 TO 88/05 1989, August
16. SUPPLEMENTARY NOTATION
17. COSATI CODES 18. SUBJECT TERMS (Continue on reverse if necessary 4nd identify by block number)FIELD GROUP Skill Retention, .Training Cost Effectiveness,Forgetting; Training Effectiveness
Skill Decay,
19. ABSTRACT (Continue on reverse if necessary and identify by block number). The Skill Retention Model is a model that describes how people forget technical tasks
in terms of such task characteristics as the number of steps in the task and the quantity
of the job aids. We administered the Skill Retention Model to seven expert wheeled vehicle
mechanical/instructors (SMEs) at the Ordnance Center School, Aberdeen, MD. A total of
nine tasks were covered, five tasks in five hours on each of two days, with one task being
evaluated on both days and one evaluator serving on both days. We identified three out of
nine tasks that are rapidly forgotten. Out of ten task characteristics, five differentiated
the three tasks easily forgotten from those that are not easily forgotten. We recommended
ways to improve training and calculated how much retention would increase with improved
training. , - I, I
20. DISTRIBUTION/AVAILABILITY OF ABSTRACT 21. ABSTRACT SECURITY CLASSIFICATION
[3UNCLASSIFIED/UNLIMITED 0 SAME AS RPT 0 DTIC USERS Unclassified22? ".AME OF RESPONSIBLE IrL-'V'DUAL 22b TELEPHONE (Include Area Code) 22c. OFFICE SYMBOL
Douglas Macpherson I 2748828 PEPI-I
DD Form 1473, JUN 86 Previous editions are obsolete. SECURITY CLASSIFICATION OF THIS PAGEUNCLASSIFIED
Research Report 1538
Application of ARI Skill Retention Model to
Wheel Vehicle Maintenance Tasks
Douglas Macpherson, Chavis Patterson, and Angelo MirabellaU.S. Army Research Institute
Automated Instructional Systems Technical AreaRobert J. Seidel, Chief
Training Research Laboratory
Jack H. Hiller, Dire,ýtor
U.S. Army Research Institute for the Behavioral and Social Sciences5001 Eisenhower Avenue, Alexandria, Virginia 22333-5600
Office, Deputy Chief of Staff for Personnel
Department of the Army
August 1989
Army Project Number Training and Simulation20263744A795
Approved for public release; distribution is unlimited..
FOREWORD
The US Army Research Institute (ARI), in cooperation withTRADOC and its schools, performs research and development on waysto achieve more cost-effective training. In 1987, ARI joined withTRADOC and the US Army Ordnance Center and School (USAOC&S) in apartnership at Aberdeen Proving Ground (APG), Maryland, toidentify and solve maintenance training problems. The partnershipwas defined by a memorandum of understanding (MOU) entitled"Establishment of a Training Technology Field Activity (TTFA) atthe USAOC&S," and dated 10 May 1987.
The present report is one result of that partnership. Thework was carried out as part of Task 344 by members of theLogistics Training Technologies Technical Area of the TrainingResearch Laboratory in order to provide practical guidelines fordesigning maintenance training which will result in a high degreeof skill retention and transfer.
This and other products of the TTFA were briefed to theDeputy Assistant Commandant, USAOC&S, in October, 1988. It willbe used to help develop model training instruction and planadditional TTFA projects at USAOC&S. Its value was recognized byMG Ball, Commandant, USAOC&S, in a letter of appreciation to theauthor. In October 1989 this project was briefed in detail to arepresentative of the Israeli Army, wh ,showed great interest init.
EDARMJ NOTechnical Director
v
APPLICATION OF ARI SKILL RETENTION MODEL TO WHEEL VEHICLEMAINTENANCE TASKS
EXECUTIVE SUMMARY
Requirement:
Support a Training and Doctrine Command (TRADOC) effort toidentify performance deficiencies among 63W mechanics in FORSCOMunits. Recommend corrective training procedures.
Procedure:
In order to identify potential performance deficiencies, weadministered the Skill Retention Model to seven subject matterexperts (SMEs) at the U.S. Army Ordnance Center & School(USAOC&S), Aberdeen Proving Ground (APG). The SMEs rated ninecritical tasks on ten characteristics. From these ratings and atable in the Model, we generated skill decay curves for eachtask. We also identified reasons for the decay. Then weinferred and recommended corrective training procedures from the10 subscales of the Model.
Findings:
(1) It is feasible and practicable to administer the SkillRetention Model to SMEs at APG. A SME can easily evaluate twotasks per hour.
(2) Three of the tasks rated are subject to severe skilldecay: replace and time the fuel injector pump; diagnose enginethat starts but stalls; diagnose engine that cranks but doesn'tstart. The SMEs disagreed among themselves on the task charac-teristics and, therefore, on the retention for remove/replace thesteering gear. The SMEs agreed that the remaining five taskswere resistant to skill decay.
(3) Out of ten task characteristics, five account for theresults above, i.e., quality of technical manuals, number offacts to memorize, difficulty of facts to recall, mental require-ments, and motor control requirements. The SMEs agreed that thetasks differed on these task characteristics and did not agreethat the tasks differed on the other five characteristics.
(4) From the 10 characteristics, we inferred and recom-mended ways to reduce skill decay through improved training andjob aiding. We also estimated the increase in retention time foreach training solution.
vii
(5) We concluded that the model could be used routinely tohelp prioritize tasks for training and to evaluate cost-effectiveness tradeoffs.
Utilization of Findings:
(1) Help USAOC&S and TRADOC decide whether to use theRetention Model routinely for training development and cost-effectiveness analysis.
(2) Help the Training Technology Field Activity produce andtest training development guidelines for programs of instructionat USAOC&S.
viii
APPLICATION OF ARI SKILL RETENTION MODEL TO WHEEL VEHICLEMAINTENANCE TASKS
CONTENTS
Page
1.0 INTRODUCTION .................................. . 11.1 Overview .................. ................... 11.2 Background ... ............ 11.3 Objectives of the Study. 2.......... 2
2.0 METHOD . . . . . . . . . . . . . . ........... 321Subjects . ." 32.1 Sujcs..................................3
2.2 Research Instruments. ............... 32.3 Procedure ........................ 4
3.0 RESULTS .................................. ......... 73.1 Application of the Skill Retention Model . . . 73.2 Data on Skill Retention ..................... 7
4.0 DISCUSSION .......................... ......... 134.1 Overview................ ....... ......... 134.2 Training Development/Job Aidnog
Implications of the Results .. ........ 13
5.0 CONCLUSIONS ............... .................... 19
6.0 SUMMARY ................. ...................... 21
7.0 END NOTES ........................... 23
8.0 REFERENCES .............. ..................... 25
APPENDIX A. Task Descriptions ....... ............. 27
B. Definitions of Scales in RetentionModel ........... .................. 65
C. Questionnaire Administration Protocol . . . 99
LIST OF TABLES
Table 1. Task characteristics rating ......... .......... 9
LIST OF FIGURES
Figure 1. Predicted engine diagnostic performance . . . 10
Figure 2. Predicted remove/replace performance ..... 11
ix
APPLICATION OF ARI SKILL RETENTION MODEL TO WHEELVEHICLE MAINTENANCE TASKS
1.0 INTRODUCTION
1.1 Overview
This report is one of a series in the Training TechnologyField Activity (TTFA) program to improve automotive maintenancetraining at the U.S. Army Ordnance Center & School (USAOCS)(Ramsay, Kessler, & Mirabella, in preparation; Mirabella, Mac-pherson, & Patterson, 1988; & Kessler, in preparation). It showshow we used the ARI skill retention model to identify critical63W10 tasks which are subject to severe forgetting (Rose, Manning,Radtke, & Ford, 1984; Rose et al., 1985a; Rose, Radtke, Shettel, &Hagman, 1985b). The report supports a larger TTFA effort to iden-tify maintenance skill deficiencies (Applied Sciences Associates,1988). It also demonstrates how the model can be used by schoolSMEs and quantifies the effort required.
1.2 Background
a. The Army Research Institute has joined USAOCS and theTraining Development & Analysis Directorate of TRADOC in a TTFAat Aberdeen, Maryland. The goal of the TTFA is to identify short-falls in maintenance performance and then build one or more modelcourse segments to demonstrate how those shortfalls can bereduced through improved training methodology. To identify theshortfalls, the TTFA representatives will observe 63W10s at workin FORSCOM active and reserve component maintenance units such asthose at Forts Riley and Stewart.
b. The results of the present study will supplement the fieldevaluations and could help us interpret the data from thoseevaluations. For example if we are unable to observe somecritical tasks in the field we could substitute data obtained viathe retention model. Or, if we find that Task X is done well atRiley but poorly at Stewart, we could turn to the skill retentiondata to find that this task is not well remembered unless it ispracticed often. The "discrepancy" in field data would then makesome sense, if we also found that Task X is performed morefrequently at Fort Riley.
c. To do the analysis, we selected the 1985 revision of the"User's Manual for Predicting Military Task Retention" (Rose etal., 1985b). The skill retention model is based upon extensiveresearch into the task characteristics that are associated withskill retention. This version of the skill retention model wasdeveloped using 78 Army tasks and military personnel (Hagman etal., 1986). Furthermore, it has extremely high validity andreliability (Rose et al., 1984, 1985a; Hagman, 1986). It,
1
therefore, can be used with reasonable confidence for TTFApurposes.
d. The model is based upon ten task characteristics thatwere found to influence skill retention such as the quality ofthe available job aids, the number of steps in the task, andseveral mental characteristics of task performance. The "User'sManual" presents these characteristics as ten scales. Each scaleis divided into a small number of categories. The subject matterexpert (SME) examines the task and determines, for each scale,the category that best describes the task. Next the SME sumsthese retention values to determine the total retention score forthe task. Finally the user refers to a table which translates theretention score inte the percent of soldiers who will be able toperform the task without training for any period from a week upto a year. With the "User's Manual" the user can evaluate taskswhich are so easy that they can be remembered without practicefor a year as well as tasks so difficult that only a few per-formers can do them properly after a week off.
1.3 Objectives of the Study
a. The principal objective was to provide explanatoryinformation on task performance deficiencies in FORSCOM units bygenerating theoretical curves of pure forgetting without inter-vening practice for the unit tasks and based on task character-istics known to bF associated with forgetting. This report iscomplete in the sense that it provides all the information thatthe reader needs to apply the methodology.
b. A secondary goal was to support a related TTFA project"State of the Art Analysis of Research on Skill Acquisition,Retention, and Transfer." We want to use the results of thatanalysis to design improved training for a testbed maintenancetask that is subject to severe skill decay. Results of thecurrent study will help us select an appropriate task.
c. A third goal was to find out if the retention model couldbe used easily at Aberdeen. The skill decay model is an inexpen-sive, quick, and valid way to prioritize tasks for trainingemphasis. We need to ensure, however, that the method does notimpose an excessive personnel burden.
2
2.0 METHOD
2.1 Subjects
The raters were seven instructors from the Ordnance School'sWheel Vehicle Department at Edgewood MD. Each instructor hadserved in an Army maintenance unit. During the interview eachdemonstrated that he knew each of the tasks that he :ated by suchbehaviors as discussing the strengths and weaknesses of theappropriate technical manuals from xemory. No rater could beconsidered senior to the others, although the raters in thesecond group regarded one of their number as more knowledgeableconcerning removal and replacement of the steering gear. Onerater participated in both rating sessions.
2.2 Research Instruments. Three sets of instruments were used:
a. Procedure lists fc: nine tasks. The task lists(Appendices C1 -C91 were prepared by Applied Science Associates(1988) from Army technical manuals, in order to evaluate 63Wmaintenance performance in FORSCOM active and reserve components.The first five tasks in the list below were selected as importantby school SMEs and school training development personnel. Tasks 5- 9 were found to be frequently performed in the field by AppliedScience Associates. Thus task 5 was considered important and wasperformed frequently in the field. Each list included the tasksteps, an indication of which steps had to be performed sequen-tially, the end products of the sub-tasks, the safety require-ments, as well as the tools and materials needed. The taskswere:
(1) Diagnose hard starting
(2) Diagnose loss of engine power
(3) Diagnose stalling engine
(4) Diagnose engine which cranks but doesn't start
(5) Replace fuel injector pump
(6) Remove and replace the steering gear on an M35A2
(7) Remove and replace the axle on an M813A1
(8) Remove and replace the clutch on an M35A2
(9) Remove and replace the transmission on a M35A2.
3
b. The Qustionnaire for Predicting Military Task Retention(Rose et al., 1985b). The Questionnaire requires that the userrate the task on 10 scales that have been demonstrated to predicthow quickly soldiers will forget task:s. The scales are: 1)presence of jcb/memory aid, 2) job/memory aid quality, 3)numberof steps in the task, 4) sequence of steps, 5) feedback, 6) timepressure, 7) mental requirements, 8) number of facts to remember,9) difficulty of remembering the facts, 10) motor controlrequirements. (See Appendix B for definitions).
c. Tables for estimating skill retention. Rose et al.,(1985b) and Hagman (1986) provide tables which allow the user toestimate the percentage of soldiers who will be able tc, performa task after periods of no practice ranging from one week to 1year. These tables are provided in Appendix B.
2.3 Procedure
a. We administered the questionnai e to two groups of fourSMEs. As stated, one SME was in both groups. The first ratingsession concentrated on the tasks that had been recommended bythe Ordnance School. The second session was devoted to taskswhich Applied Science Associates had observed were beingpe,-formed in the field. Task 5, "remove and replace the fuelin:ector pump", was rated by both groups of SMEs.
b. We interviewed each rater independently. We gave therater a questionnaire and the "User's Manual" and then "talked"him through the ratings. The task analyses and the questionnairefor predicting military task retention were amplified beyond themanual, according to the protocol in Appendix C. In addition wealso "highlighted" the most relevant sections of the User'sManual so that the raters could find the information defining thescale values more quickly. At the request of our SMEs, we madetwo additions to the protocol. In response to our experts'uneasiness at rating recent graduates we defined our mechanic asone with two years experience. We standardized the location ofthe operations by specifying that all maintenance took place inthe shop rather than in the field on maneuvers.
c. After the SMEs finished their ratings, we assembled theminto a group. The group then discussed each rating of each taskuntil it reached a consensus or seemed unable to do so. If itfailed to agree, we used the modal value, as required by themanual. Three situations occurred where we decided to presentthe disagreement among the observers. We present the two sets ofdata that were produced for these three tasks. We describe thesituations producing the two sets of data below:
(1) In the "engine cranks, doesn't start diagnosis" the fourraters split into two equal groups and were unwilling toshift. The manual states that both sets -f results should be
4
used.
(2) In the "remove/replace steering gear" task therater considered most expert by the others scored thetask on the basis that the job aids (manuals) would notbe available on the shop floor, then was unable toattend the group session. We include his data becausehe was considered most knowledgeable, his responsesdemonstrate the predicted effect of lack of technicalmanuals, and they correspond with the informalobservations of an Ordnance School field observer.Furthermore, they agreed with those of Schruman, Joyceand Porsche (1980) who, in their study of Army wheeledvehicle mair. anance, reported that TMs were unavailable,obsolete, and that "mechanics who did try to use themanuals seemed unfamiliar with the layout and indexing"of them.
(3) The "remove/replace fuel injector pump" was ratedin both sessions because it was selected as important byschool personnel and performed frequently in the field.Thus it was rated by the group of raters who rated theimportant tasks and the group who rated the frequentlyperformed tasks.
5
3.0 RESULTS
3.1 Application of the Skill Retention Model.
a. Total labor for application of the model to 8 SMEs was26 hours or 2.8 hours per task. Total clock time was 10 hours or1.0 hour per task. Each rater, working individually with aninterviewer, took an hour to rate the 5 tasks, i.e. 12 minutesper task, per rater. The group discussion, led by the inter-viewer, took another hour for each group, i.e. 12 minutes pertask. Interviewer labor was 12 min per task, per rater forindependent sessions, and 12 minutes per task for the groupsessions.
b. The equations below were used to calculate the totaleffort required:
(1) SME labor (individual) = 12 min x number of tasks xnumber of raters
(2) SME labor (group session) = 12 min x number of tasks xnumber of raters
(3) Interviewer (individual) = 12 min x number of tasks x
number of raters
(4) Interviewer (group session) = 12 min x number of tasks
3.2 Data on Skill Retention
a. The retention data appear in Table 1, which is a taskscoring sheet. The data's implications for task performance arepresented graphically in Figures 1 and 2. The table gives thegroup ratings for each task and questionnaire item. The tasksare presented in the left column in two groups -- "Diagnostic"and "Remove & Replace." Total task ratings are shown under"Total Score." The numbered columns identify the scales. Thelabels for the columns present the scale values and instructionsfor skipping certain scales. The scale value assigned by thegroup to the task is on the row for that task. The cell for acolumn is filled with a dash when the instructions require thatthe scale be skipped. The right-most column translates the totalscores into the time lapse since last proficient performance thatis required for 50% of the troops to forget the task. Thus forthe task "engine starts, but stalls" half the mechanics would beunable to perform the task to standards if they had not performedthe task for about 6.5 weeks.
7
b. Note that group ratings varied by one scale step or lessfor scales 1, 3, 4, and 5. Technical manuals exist for all thetasks. 1 All tasks have more than 10 steps. All tasks aresequential or mostly sequential. All tasks were rated easy toperform within the time allotted. Scores for the remainingscales, however, varied considerably, e.g., the range of valuesfor amount of feedback is 0 to 22. Scales that differentiatedamong tasks are listed below in order of decreasing effect onretention.
(1) technical manual quality,(2) mental requirements,(3) feedback provided by the task,(4) number of facts to memorize,(5) ease of recalling the memorized facts, and(6) and physical skill required.
We identified and rank ordered the preceding scales byexamining the columns of Table 1. We observed that the tasksdiffered by large numerical values on the scales listed above, andthat these values represented adjacent scale categories.Examination of the Performance Predication Tables in Rose etal. (1985b) indicated that differences of these magnitudes wouldincrease retention times by at least 50%, therefore we consideredthese scales to identify possible areas for training intervention.
c. The task retention tables in the User's Manual were usedto create two graphs presenting 12 retention curves for our ninetasks. 2 The results are presented in Figures 1 and 2. Figure 1shows that skills for low to moderate scoring tasks decay veryrapidly compared to those for tasks with higher scores. Forexample, at one month the passing rate ranges from 90% for "enginehard to start" down to 33% for "engine cranks, not start 2" eventhough the soldiers were able to get a "GO" on all tasks a monthearlier.
d. Another way to see this difference in decay rates is tofollow a horizontal linc from, for instance, the 50% point on theY-axis, mark where that line crosses each of the retention curves.Figure 2 shows that, without practice in replacing the fuelinjector pump, 50% of repairmen would lose that replacement skillin only 2 months, according to the first group of raters. Incontrast it would be 8 months before 50% of repairmen forget howto remove and replace an axle.
e. Tasks achieving scores above 180 are not forgotten in ayear, the maximum time spanned by the tables in Rose et al.(1985b). Since removing and replacing both the clutch and thetransmission scored 180 or higher, the curves for these two tasksare plotted as a single line across the top of Figure 2.
8
06 N
en wl m %n In+1c
en- C4
-- - - -
Ue)V t - - -
Vi-
~ oo
~c Cu(.4
m 0 n
= n nWcnmm m lM ne
co~
U)0
Cu 4
-C) - --- -
C4 14 ) n C C
r4 ~ ~ CIAC1
oz -m~ >.x sd aC
5~ c% E ob I
~., -, * 0 ~E
9
a)
EI-0
o IL.-~ 0)
0 0)
.4, .
0)m 03
o3 (00
Lrn .4-1
-~.m(a 0
u C
-4-.0 0)
CD
o L-4~ L
0. a
U-
a <M o 0 N co m c, 0 o o
uoujauo o4 Buiwiopod %
04.,
0
E(fl L-
n 0)
I- 0L o-
-~ E00
CP :3
L. a)
va)
00
-C)
C14.
oo
0)
11~
f. As described in the procedure section multiple scores arepresented for three tasks:
(1) The task "engine cranks, doesn't start" two SMEsbelieved that the mental requirements (scale 7) best matchedthe "simple" examples in the manual, whereas the other twobelieved that the average 63W mechanic found the mentalrequiremenit more like the "complex" examples. Although theseresponses were adjacent on the rating scale, tasks withcomplex mental requirements are retained only half as long asthose with simple mental requirements.
(2) "Remove and replace fuel injector pump" receivedratings from the two rating groups. The two ratings differprimarily because the first group believed that the taskprovided feedback only at the end, when the mechanic testedthe pump. The second group considered that each stepprovided feedback, that the mechanic could tell if he hadmiss-threaded a nut, for instance. Thus the differencebetween these ratings is a result of differing definitions ofthe scope of feedback to be considered in the evaluation.
(3) In "remove and replace steering gear" the most expertSME on that task stated that manuals were not used on theshop floor. The remaining SMEs scored the manuals"excellent" and stated that they were available to themechanics on the shop floor. Thus we present the "withoutmanual" results with the caveat that they represent thejudgment of one individual rather than a consensus. Asstated in the procedure section other observers havecommented on the lack of manuals on the shop floor in thefield.
12
4.0 DISCUSSION
4.1 Overview
a. The results show that it is feasible to use the SkillRetention Model as a tool for differentiating and prioritizingcritical tasks at the USAOCS. The model can be used to identifythose tasks most susceptible to skill decay and therefore most inneed of special training resources. Given its high reliabilityand validity the model provides a very cost-effective supplementto field studies since the current procedure can process 1 taskper clock hour. A similar number of tasks, studied in the fieldis currently taking about 8 months, at a cost of about $200,000in contract funds, in addition to extensive labor by TRADOC andARI personnel. The model, however, is not a substitute for goodfield evaluation, which reflects many factors other than pureforgetting and which provides details not available from themodel. 3 However, it is a critical supplement, since it permitsthe user to quickly evaluate large numbers of tasks.
b. This application of the model demonstrated that SMEscould agree on appropriate scale values a large proportion of thetime, since they agreed to disagree on only one task of the ninestudied, "engine cranks, doesn't start." The results of the twoadministrations of "Remove and replace fuel injector pump"suggest that intra-group agreement on a task does not rule outdifferent ratings by different groups. Thus, when multiplegroups are being used to rate tasks some of the same tasks shouldbe rated by all the groups, so that any biases can be corrected.The correction might consist of immediate retraining of thediscrepant groups or post-hoc adjustment of their scores, therebymaking all group ratings more comparable.
c. The results also point to clues as to how to overcome theperformance deficiencies predicted by this study. They do soby identifying potential task and training variables whichaccount for low task ratings, and especially variables whichdifferentiate among the tasks studied. We discuss these cluesfurther in the next section.
4.2 Training Development/Job Aiding Implications of the Results
a. Job Aids.
(1) Problem
(a) A major difference among the tasks was the varyingquality of available technical manuals (TMs). TMs don't helpmuch if they are hard to search through. If they were judged
13
"excellent" then the task would be performed correctly for twiceas long as the same task with less than excellent manuals. Thecurves for the "remove and replace steering" demonstrate howimportant available excellent manuals can be.
(b) The major difference among the TMs' effectiveness forthe different tasks was in the number of manuals required toperform the task. The low rated tasks tended to requiresearching multiple manuals to do the job whereas the other tasksdid not.
(2) Training Solutions.
(a) Increase the amount and quality of training in how touse TMs, especially where multiple TMs are required.
(b) Place even more emphesis on requiring 63W students inPhases 1 and 2 to use manuals in all aspects of training. Givemuch more experience in the tasks requiring multiple manuals.
(c) Develop special aiding techniques for tasks requiringmultiple manuals or very difficult searches through the TMs.e.g. special checklists which pull procedures together in oneplace. (See Swezey (1987) and Elliot & Joyce (1971) for detailedguidance). Use the results from applications of the skillretention model to identify the tasks most in need of special jobaiding.
(3) Example of Pay-off. For the "engine cranks - doesn'tstart" problem, the 50% failure rate point could move from 4 daysto 3.5 weeks with improved job aiding. Excellent job aidingplaces the memory in the aid so, essentially, the job is notforgotten as long as the job aid is used.
b. Mental Requirements
(1) Problem.
(a) We could double retention by doing a better job ofteaching students to analyze maintenance problems moreeffectively. This is, in fact, the goal of the Basic Skills andKnowledge (BKS) training, Phase I of the 63W10 Program at Aber-deen. But, nothing the ARI researchers have observed wouldsuggest that Phase I training has a marked effect on task perfor-mance or retention (Ramsay et al., 1988).
(b) The "Users Manual" provides clues for solving the"mental requirements" problem. It describes simple mentalprocessing as "making gross comparisons ... or performing simplecomputations" (op. cit. p. 29). "Complex mental processesrequire the soldier to make a choice ... based on subtle but
14
discrete clues (e. g., identifying different types ofaircraft or vehicles)" (op. cit. p. 29). Thus the manual impliesthat the students must be taught how to discriminate, generalize,and reason about situations so well that the behaviors requiredbecome as natural as gross comparisons or simple computations.
(2) Solutions:
(a) Front-end Analysis (FEA). Apply the basic principles ofFEA described in the Harless method (Harless, 1982). In par-ticular identify where soldiers must discriminate or generalizeto perform effectively. The Army routinely ignores thisprescription and describes tasks as lists of chained procedures,an inaccurate and misleading approach. We can't begin to solvethe skill decay problem imposed by "mental requirements" unlesswe define those requirements accurately and operationally.
(b) Apply special techniques and pay special attention totraining discriminations and generalizations, as explained inHarless (1982). These techniques included isolating these partsof a task, using verbal mediators or shaping techniques fordifficult discriminations, and providing added practice. Forinstance it might be possible to train soldiers to use learnauditory diagnostic cues to the level that these cues would be asnatural as gross comparisons. Among the target cues might beuniversal joint, differential, and brake noises. We suspect thatauditory cue training might be useful in adjusting the fuelinjector assembly.
(3) Example of Pay-off. For the "replace fuel injectorpump" task improved methods for coping with "mental requirements"could move the 50% failure rate point from 2.5-8.0 to 5.0-16.0weeks.
c. Number of Facts to Memorize/Difficulty to Remember
(1) Problem:
(a) Number of facts contributes substantially to skill decayfor the three tasks involving engine starting problems.Difficulty of remembering facts is an added problem for acranking engine that won't start. Number of facts is not a majorproblem for the remaining tasks, although the few facts connectedwith replacing the fuel injector pump were considered a bitdifficult to recall by one group of SMEs.
(2) Solutions:
(a) Good job aids can do much to off-set this problem. Thekey however is to make the facts easy to get at. The SMEsstressed that facts buried in multiple TMs are not easy to find.
A small well indexed card catalog with the most frequently usedtolerances, settings, and diagnostic readings may be one answer.The Skill Retention Model can help identify what set of factsshould go in the catalog.
(b) Training solutions.
(i) If a task has 8 or more facts, pull them out in acondensed list and give the trainee practice in reviewing thelist. Teach three or four facts at a time. Do this before thesoldiers practice the whole task, and then again after theyfinish the task. If the same facts are used in other tasks,prepare a list of those tasks so that the trainees can see howthe facts generalize. Show clearly how the same fact (e.gtolerance, spec, formula) applies to other tasks.
(ii) Use memory aids (i.e. mnemonics). This requirescleverness on the part of the instructor to generate mnemonicswhich make sense and are easy to remember for these particularstudents and topics. e.g. the instructor may need differentmnemonics for different types of students.
(3) Example of Pay-off. Successful efforts to reduce theskill decay effects of large numbers of facts would move the 50%failure rate point for diagnosing a stalling engine from 6.5 to9.5 weeks.
d. Motor Control Requirements
(1) Problem. The "User's Manual" and its first author statethat tasks requiring a considerable amount of motor control, suchas that required for touch typing or driving a manualtransmission car, are retained better than other tasks of greateror lesser difficulty. Some of the tasks, "hard-starts", injectorreplacement", and "Power loss" require considerable amounts ofmotor control, the optimal level for task retention. Othertasks, such as "remove and replace clutch", "engine cranks/nostart", and "engine stalls", do not.
(2) Solution. Consider increasing the complexity of thephysical performance demands of the task during training. Forexample, require use of torque wrenches and other precision toolseven though they may not be required or used on the job.
(3) Example of Pay-off. For the "engine stalls" problem,the 50% failure point would move from 6.5 to 10 weeks.
4.3 Implications of the Model and the study for cost-trainingeffectiveness analysis (CTEA).
10
a. The examples of pay-off suggest that the Skill Retentionmodel can be a very useful tool in doing CTEAs, i.e. in answeringthe question "how much is a pound of training worth". Forexample, we can, with relative ease (e.g. see Adams, 1986),estimate costs for any proposed training solution or solutionsderived from the model and then estimate from the model, thenumber of weeks of retention provided by the solution.
b. For example, the model suggests that a 50% pass rate forreplacing the fuel injector pump can be extended from 2.5 to 5weeks (worst rating) if we provide job aids that help themechanic move through multiple TMs. We can estimate how much itwill cost to develop and implement the job aid or evenalternative job aids. Putting a value on the extra 2.5 weeks isnot quite as straight forward, but with some imagination it canbe done. Look at the cost of retraining, especially for reserveunits that meet once a month. You can then make statements ofthe form "a dollar for improved training will save two dollars inretraining".
i !1•
5.0 CONCLUSIONS
5.1 We conclude that the Skill Retention Model can:
a. be administered easily and at modest cost to APG SMEs,
b. predict skill decay for wheel vehicle maintenance tasks,
c. identify tasks that will be forgotten most quickly,
d. identify task features that promote skill retention,
e. help define training to reduce maintenance skill decay,and
f. support cost training effectiveness analysis.
5.2 The Skill Retention Model is sensitive to differences inhow fast 63W skills decay and can be used with 63W SMEs.Furthermore, prior research has proven that the model is reliableand valid. Therefore, we recommend that
a. it be used to rate the remaining 63W tasks, and
b. that its use in CTEA be exanined.
I k)
6.0 SUMMARY
6.1 The questionnaire provided in the "User's Manual for Predict-ing Military Task Retention" was completed by seven wheeledvehicle SMEs at Edgewood Arsenal, working in two groups. Onegroup of SMEs rated five tasks that had been selected by schoolpersonnel as important. The other group rated five tasks thatoccur frequently in the field. One of the tasks was rated by bothgroups since it had been selected as important by the school andoccurred frequently in the field. The raters were able to use theinstrument to rate ten task characteristics. The instrumentdiscriminated between the tasks and predicted large differences inthe retention of different tasks. The results of the ratingssuggested that 63Ws may need much more frequent refresher trainingon some tasks than might be expected. The descriptions of therating scales provided in the "User's Manual for PredictingMilitary Tack Retention" suggested approaches to revising thetraining that could increase retention.
6.2 We concluded that the Skill Retention Model is sensitive todifferences among 63W tasks, that it can be used by 63W SMEs, andthat it can help identify job aiding and training procedures toreduce skill decay. We also concluded that it could be a usefultool for cost-training effective analysis.
7.0 END NOTES
1. The steering gear replacement is not an exception. Onerater stated that they exist but are not available on the shopfloor. The reason for including this rater is explained in thetext.
2. The reasons for two sets of results for some tasks wereexplained in the procedure section.
3. For instance, the initial analysis suggested that thediagnostic tasks should be critical. The field research failedto observe them and determined that the remove and replace taskswere critical in terms of frequency of actual performance.
2-)
8.0 REFERENCES
Adams, A. V., & Rayhawk, M. (1987). A Review of Models of Costand Training Effectiveness (CTEA) Volume II Cost Analysis (ARIResearch Note 87-59). Alexandria, VA: U.S. Army Research In-stitute for the Behavioral and Social Sciences. (AD A189 645)
Applied Science Associates. (1988). Data Collection Plan: Im-provement of 63W Wheel Vehicle Repairer Job Performance. Wash-ington, DC: Applied Science Associates.
Elliot, T. K. & Joyce, R. P. (1971). An experimental evaluationof a method for simplifying electronic maintenance. Human Fac-tors, 13, 217-227.
Hagman, J. D., Hayes, J. F., & Bierswirth, W. (1986). At thesquad through company level a method for estimating task reten-tion: Research pays off. Army Trainer, Fall, 54-55.
Harless J. H. (1982). Notebook for Workshop on Front-End Analy-sis, Job Aiding, and Course Design. Newman, GA: Harless Per-formance Guild, Inc.
Kessler, J. (in preparation). A Discrete Event Simulation Modelfor Determining Optimal Flow of Classes Through the 63W AdvancedCourse (ARI Technical Report). Alexandria, VA: U.S. Army Re-search Institute for the Behavioral and Social Sciences.
Mirabella, A., Macpherson, D., & Patterson, C. (in preparation).State of the Art Analysis of Research on Skill Acguisition. Re-tention, and Transfer (ARI Technical Report). Alexandria, VA:U.S. Army Research Institute for the Behavioral and SocialSciences.
Radtke, P. H., Rose, A., Shettel, H. H., & Hagman, J. (revisedJune 1985). User's manual for predicting military task reten-tion. Washington, DC: American Institutes for Research.
Ramsay, D., Kessler, J. J., & Mirabella, A. (August 1987). NeedsAnalysis of the 63W10 Course at Aberdeen Proving Ground (ARI Re-search Report). (In preparation), Alexandria, VA: U.S. ArmyResearch Institute for the Behavioral and Social Sciences.
Rose, A. M., Czarnolewski, M. Y., Gragg, F. E., Austin, S. H.,Ford, P., Doyle, J., & Hagman, J. D. (1985a). Acauisition andRetention of Soldiering Skills (ARI Technical Report 671).Alexandria, VA: U.S. Army Research Institute for the Behavioraland Social Sciences. (AD A140 336)
Rose, A.M,, Manning, C., Radtke, P., & Ford, J. P. (1984).Acquisition and Retention of Soldiering Skills: Report of Year 2 Progress.(ARI Research Note 84-85). Alexandria, VA: U.S. Army Research Institutefor the Behavioral and Social Sciences. (AD A142 436)
25
Rose, A. M., Radtke, P. H., Shettel, H. H., & Hagman, J. D.(1985b). User's Manual for Predicting Military Task Retention(ARI Research Product 85-26). Alexandria, VA: U.S. Army Re-search Institute for the Behavioral and Social Sciences.(AD A163 710)
Schurman, D. L., Joyce, R. P., & Porsche, A. J. AssessinQ Use ofInformation Sources and Quality of Performance at the Work Site.Valencia, PA: Applied Science Associates.
Swezey, R. W. (1987). Design of Job Aids and Procedure Writing.In Handbook of Human Factors, Gavriel Salvendy, ed. New York:Wiley & Sons.
26
APPENDIX A: Task Descriptions
Diagnose hard starting
Diagnose loss of engine power
Diagnose stalling engine
Diagnose engine which cranks but doesn't start
Replace fuel injector pump
Remove and replace the steering gear on an M35A2
Remove and replace the axle on an M813A1
Remove and replace the clutch on an M35A2
Remove and replace the transmission on a M35A2.
IIIIl I
: !I - •
I Ii
300
I i i C
I. dLLI
I IS
mc0
z
r v
iW J .I•I -
i
I -O.I
a I &3
- II
o I __ dc _ I _ _IV
"hE • m *,= •
; i •,• hi IU"• = +i
r- c MB
(ii CCa 4
28 .'a h
I. f
90 41
00 6 1
I~W I :!- a
W. ii = i .;
-At
4 I00oLL -2
IW -r
LU C
:~U Go 3 I I o
C.. -.
LU 6 6. (.0 I~ w a
to Ml= r, - 619W. VA 4C - r- W It
Lfl~~~ A-I - - .
j~ Ii I
29 ............ I... 1
LLI
z
0 ~i~
-- m -. _ _ j 6,
th 6
I I30
C.,4
-4 Co
I 0 ~ 'JNCN
u -
0l. (.OC) 4J .M
0~* 0 Wz
ca E 1-- -C c - -- - -=)~ ~ I Ima)c> d
-c -P - -
0 l~0
to 0 V4 -r4 l5a C
3a Cu.' c 0 " 4 j1-4 CL V4 *0 m
Jo 0 u .- 4 u 0
a -4I
.1 0
-D4
I- '31
En F- I I I I I I
0 z
ZLJ
O~z) -. 0
u. Ia
W) 0 0
1 0a 0) 0
rz 0) o(-Cu 1 0 0)0 +0j4 6
0 Ai &a dCJU 0.0)C U CD M 0)A o .
z w ) w -4 4) w E-t40 40 ) m-
- W-44 - -L - -1 1- -j -4 0- V
>' -- (I. Q. In -m
4) u-. -4 -4)I 4W c mO (n H 4. fI 0 ca 4)z m 4- N 1
(w ýt 0 0 a0 -W0a.4) w-H ) : 1-4 0o to-L -4 -o I
5 -'0)'-00) 00CLk CL m4~' L. C H, M CO IM- C; 04I' -I 9 e
COI. ~ U0 fl) C~.J Cl )4J32O.~
z :D
I~00
zE W
04
Q) - -4 Z
- -> u E
C "0
Q) 0)0
U-U
AA V
Q) CU31 C4 f
-. -j -0.& cý -1- - -u -= ý1 ' -401 w ,- u0 0 H cU - 1 - 4
m - - - - r. :I-
z > > E -- > .0 m u - = Ur'4.00 0 0"a2~0 0 u. 0- 0-4
- ~i -I- Lro= 0.0, .
LL - t a - - . -
Suz
.44 C1
Zwz -00wC
4 U
Q).64.-
Aj-
0 .6J 4 .-j
60 0) '- 0.'-
* r.4 -4 0 ~ (n, )C
0J Q) wu 100- u CLc70-
0 w w w a) 0. 0.r ucv - - -a - -u-j4t
0 - - cc -dWzwa -1 .4 t
-4 a aA E>aV. .0
oo Aj~ m ~ 0 U.tza> ý) 6 4 k;:,. ~
ODI
*1 5 0 I
0I 0)0 w -4
Ij .9W Aj 41-4 r.4) 4 w-40
0U ý
O' 0 -
0 4) 4) >.3 0- 4 9 ý
cc 0-1 W 0 0 U=~~ 1-44W. 2: O
.0 =0 0 )0 u V m
oo 41a3 C . 010
W to a ~U~ 1-4 4) 00w boW r
0 A"W -4 -r4 M0) -
W ol 01. 4.5 co
cW z 0 "4Wi 01 CO) 0C r_
Wm -4 Z 1.4 00 6O.
0 41
cc cc 4 2~ to COO4
m -L - - -. v -o --cnaibo
.4 W -4 CJ0 4
.4 m to 4 4) c 4)0 u t hV 1 w M 0.
07 L 1Ii. '1 01 1
00
Ln 17
o LL1
0 zj
m 1 0
te0 "l ri0eLj 2j o
SE .0ý3~~~ C.~ -0
M~ 0 aC
V2 V) w ) C c 0 -4 V) Q
CO (0 4AJ-) 4 -4c -4 Q) co-ý4En 4 -4 " f cz tv 2) 2)u . t
CL CL- w: c .0 C) W~ 0 ) 0 o Q
> 0 tv'- r-. - C: -4 w-H ýd 2c
-,4 uco~~ ~ ~~~~ w( ) wa 0C:Q )4
-~~ -r - -- - -A
0 -uz0 ý4 0)C
II 0$
:R -I >0
1J 0 r )c
-- = - -4-4 M
LI :C) tol CLaw cc. mf Ul N H4
c6 00 r- a) 97
- i z -
-C- Uw 2>1E
"-4 -4~.7- E --- 4 'V- X >H -4 4-4 r_ C0 m
ci w ()(
41~ -H - - _ _-jca (
0f4 w ) - A Wco > w ~ 4- 4-1 C6~ w-
0.Vo $4 (VO Z-.> S~ 'n oE U-4 4- wJ T- 00) 0tCiQ 2JE cu. CO U)U Q)J. CQ0
V) J.a w f~ -! 0 -
r w 0 0 F. F- 4
z .61 A x h -
- ~ ~ ~ ~ ~ c 0 l
E (n
LL.~ 0 CL
a0 QJ -4 Q) Q1 ý4 E~ Q).
I 0W 0 C~Eu ab ~~u'.t ý -0- 0 -
w $..~ 0 CO
U- -0 E
-n C
0-Cl
* r. 04
U~ wI
0 14 L" 1 0.0 rn A 'un
Wt w -H (n
m ;x. w 3: c~ 0 w
o (0m comQ
0- O-4n 4 :g 10z~~~ b,.40 C 40 0
38
l l I l I I I l I
l-z"I II !
SGoV cc a*
° Id
'U! I-.- 0
S!L _I l I ~ -' iuus
I. III .I M C
be9
U ii
ASD.
o I1] n- . - . - . - . - . - . a
31k b. 6u ". 41 ,.
U I.
h'. . C.1 a
iI I I
~ii "I i
i
I I I
j~t Or.= 6
urn ,-, ' I•z I}
" I G D. [ .
cc v a.
- S ,-, "-. w
> 1I. ___ A .:B ii1- -- l tIW -00.
"a M t v - - a - r .- . a
, Il'' _ Ilm-l jl l I.-• ..t ! ,,I
W W h. j , a CWC i IV . - - -
IV. a, Mj 6.oU4
L.L.
* CO
v- 4,r-4I :1
:1 IUIUu
Iri v____
41A
'I Iz'I I a
CC
%d Z
j at i.
cc a P.at v Id&
.1.~ ~~~ 4.4jI' I ~mEn - - - x ~ 1
6. Ii I I I I I I I 42
c El
oe I I
1 i
a, ". a
I -Crow -a (
433
I I I-
LL
" I _ _ _
S° Ii
0 ,
P v 1.. Z `4 " &. .
A 05 •I- 0S
, £ U IC Si um x
I IC Im 0
I.. -t . 4. &il
*0Si
""m C- '.. .t C
K1 1 w .1 amm 1' .m
4~~ %. C! C . C P
44. a ~ as a
'I I',-I I I
V I I1
V -
- I _ I _ _ _ _
S. I ,
Ij to so
i- 4ac...•, 4 w
ad IN b. -A 6
ad %- 4 9
~2 c
~j, .A - - - -45
I Im
4iI, of 40"*~ - a.- -
of VII t
a) v1 . C 1w
(AqJ
c 44 6, U4 W U
toa jGl ds a C4c 4,"
ci zo -- wzL2 F r 9& d43' . Ill, i7 iiV.,m
c)f3
46
iI ,
1'- .
ILL
zf0I ~.1
Li
- f- ._ _iiI is _*!1 - ..CI S
""' : C,
SD 'a -c ' : Si S
60 o - 0 -I , O -
I I aI
& I I) w I0 -&4"a
F.F0 r
A47
I J 9I g i
m -i
VII z41 W
CII.rI Ai C)w
ii *C2cv
48
I CNN((nI I cnC.. -17-:r u
NCN N -I -Ir-4U a I
I I I IN N N a) Q)
fn00 E -4 0 cou
INN I -10. 0) w
a0 - - --
0
a0 )
Aj .0 w E--
- p - - - --q -) 1 .
o ( A a) 0 C uaI- C:W r) r- -
* 0) r_) U 0 a40) 0
~ 0):0 0)C 0)tl-40 U 0 U U 'IV Z~ ~.6 -
-,- 0 .)
LL c c
(fl cuS --
.14
mam
~J C,
cu
Ur 0)C
-- 4 0
co0 0)U$4. 0 )V
4 9
LI!I
o -. c>I 144 .60 u:H )S -a aW044 i 6 -
c - )A)0 . -- n 0a z ow w> uw Q
0 ) -4 00a .. >w > >u aZ j 0 00 00Q00w0 0 C W
I-- -9 t9 q4b1a 1
0i
J a)0-~~t
M -1
SUSU ESý --
Wh .1O-4 W w a
EC (** a)- WE ,
E-4E U. > ~ E
0~~~~~~ c c0(1 C3rt
4= v C I ) c - Ic o ý -H u 4- 1
u~~ ~ ~ ~ wOQ W1 >C
0 w-to a a - -0 -, m > 0 c
< ý-4 ý- ) m - ) 11ý
-z w> ca E -
2) w0 0 2V51 q C14ý
I $4 .13
Li.]
*~ 00 =-4J Q
~~1- (A -4- - - - -H- -
o. -C 4L -IL -L
SS
LLiz
0 I
C9,
uui
o.. -I
*oI96 (vQ - . ---- a W-
C: V)- -- - - - - - - -
ozn V) w( n too o
,..-,. t,
P cww-S- --- -I - •-- ,-, , - -, -, - -
i5
-•- - - -I-I- -I
X 0- -H a -- - - - -
0 4Q 0 0 0) 0
AN Q) C4 C%4 C14 eq C1N (Ni
U >0) allal I c I7 al ol al ,Il
0 0000
zz
> 0 1
I t
Ia - -1 -4 "1-
-04 00u>
0 0) Q ) ( 0 w a)-4
u4 > - 0) h0 Q)k ho H b
f~1 0)u HZ M~l 0 )
.544
£ I I -I-i-- - - - - - -
M C4
S 0CN
00 c
LLt
- -•
LUU
OD S•"-I L ,
,.- L. C: L c ) 14U
• no
r-- 41` i _1. 1- 1 0 -4 a u 0 e 4- - ) t 0Q - -
co .2 00 w0 1 0 m -Q) ~ 0) w a) m 2 )4 =a 4 Q)E wQ )0 u> H j >r4 > : >-4 C4 > 44 > P >,- .4~0 u00 H 0 ,0(na 44 0 0 0 c 0 4-4 w Q
a) u a)O A) w ) a C V )Qr_ Q) 0ý :L 0 Q) w w -W() ob0) L) W0)-2)- ) J. 00 U.
.w w I
Zb >
ra 0
S 0 r.0 jýci a 0$4 O -
co 41b3c orS 00 C4 0 a w-HQ
- 4.- 4 -H - - 4 -4 Q)- -
0000
Ai4 .4Ai0 m S i u 000> -a w ~ CL-4 co8 -4u00()0 41 A.,. S-H 4. )c-4~0 ~ .CA j S wJ~ 0I-i~06 mO4~ > t4-0'4 t0 U2 #-4,4 4 4.j U2-.4 -W u4 *: e (Y.I 0 ~ . ~ -48 ODi -4C j 0U ~ -3 .Cj~0 04 c~ 0 04 0 jc i 2 0z-r 0 * OS dS i 0 0 0 . 0OA ~ ~ ~ ~ - r 0 l .0 m - -- ~- .
50 4 ~ 3 0
- -~e -4 - 1
CO ca (U e.. C1 ' - - -
o o W
40) w Z
0 m -1 = -C
LL~
zI
0I
-J
-b -H - --
> - 0 - w a)
u 0
U) w u ) 0) w )wm. C- ) >4 >. >4 w > > 4
(D - -d 41) 0 G0 C 0) 0~ C)
0) m~ 0 ) n) 3w OU
:x "~ "U 0) oz -ld w0 ) o) 0 4. 0)0
0~ 4.' .. 0 0 . 0rl l r 0)1 nS- :1 r'i-4 M 5.0
USU
U>
ii wSr 1 -U) m14 4 -
V. w- to L n 30 1 3 I-& 4 00w ) 0 u 0- 414 1UIaS 4 - ) - ldz u =x ) r u Q ,
I - - -----------------------------------------------------------
V -
9 -4
11 1~~ V 7
U-4
"eli
UJJ
- -) aQ a ad 0
0 M
-~4 ' aa -C-
LU4 00 &JA U n 0 Om .04 .$ : rod4 114 0 t4 0 -
rnX(cu) co r(
W i 1: n
Lii. -4
U.U
S ~I
SQ90 r
oca
I: 1- 00wc 16=t - 0
I I
s -- -l r- u -n - = -) ( - - -tZu
o
W :•
60
cn lIfillII liI I
Ii ;
00 r- u a w-
-4 4j 1 1czQLi -) - w -4 --
w -4
9- r21- u ()c30 10 1 1M wW
m Jd = LN . . . .
lII
0 w
I.?:U. - 3 E- 4
0
iN
zz00
I:"MI
p..
ri w w, Q) JýP : 00 • P 0 OD(:.,-4 ' z ,,-.-w > a) O U 0 , z- m
u C2 " . 0 A j : U) 4. 44•
Q.) 0 -H w 4) u "
00
wI Q)0 >(
cu I -I !E tor) E0 E0- 0 -4 - w -4 -a)4 -1 $4 w-s
L. -. _- ,- OrC 4 -- x 4 a,
:3 0 0)
020
cc•
A1
I41 ý4 I - - - - $4
V) z :u1- . QI.4 U)0
Ii i
- - -a - . 0 14$4 - 4 --4 . -4E0 a) m 0.C ., a L
mo
w 0
o a!
-,,,'- -
- • -- •-•- -- mmn Imm Im n ll nll II I I I • i6
APPENDIX B: Definitions of Scales in Retention Model
(1 5
It. Instructions for the Paper-and-Pencil Version
The task rating method has been developed in both a
paper-and-pencil version (described in this manual) and a
computer-based version. This manual is intended to serve as
a basic guide for those who are asked to rate tasks to
determine their level of retention. While it contains the
essential information needed to use the rating method
properly, situations may arise for which adequate guidance
is not provided. Further assistance may be obtained from
the U.S. Army Training Board at Comm (804) 878-4658 or AV
927-4658.
iT -The Rating Procedure
The task rating procedure contains ten questions with
•I each having from two to four choices of answers. In
addition, there is a Definitions section designed to clarify
U. the meaning of each question and help with the selection of
the most appropriate answer. It is important to read all of
the Definitions information before selecting an answer.
* The paper-and-pencil version contains a Task Retention
i Rating Form on which the answers to each question for each
task are to be recorded. A copy of the Rating Form is
I included on Page 40 of this manual.
T00
Here is the step-by-step procedure to follow:
1. List the number designation and title of each ofthe tasks you are going to rate in the first twocolumns of the Rating Form. Use one line on theform for each task. You may shorten or abbre-viate the task title. Enter the MilitaryOccupational Specialty (MOS) or Specialty Code(SC) designation at the top of the form.If you are rating tasks in more than one
7 MOS/SC, use separate Ratings Forms foreach.
2. Refer to the current task surmaryL[ to obtain a description of each task
you intend to rate. You may alsouse information found in referenced documentsfor each task.
3. Read the first question and the associatedDefinitions section. Review the supportingdocumentation for each task as needed toarrive at an answer. Note the Scale Valuefor the answer you selected. Write thatScale Value in the box in the Rating Formcorresponding to that task and question.
4. Continue in this manner until you haveanswered all 10 of the questions for Task 1and have entered the appropriate ScaleValues on the Rating Form. Depending onyour answers you may be asked to skipcertain questions. Follow the instructionsgiven for each question.
5. Add the individual Scale Values for Task1 and enter the total in the "TotalScore" column of the Rating Form.
6. Follow the same procedure for the remainingtasks you wish to rate.
S7. Review the ratings given to each task by otherraters. Resolve differences and document anychanges in or assumptions made about a task.When the scale values for each question andthe final score have been agreed upon by theraters, record them on a separate Rating Formnoting that it is the "Approved Task RatingForm." Supplementary documentation should beattached to this Rati,.- Form to support the[ ratings given.
[ 6•
In order to convert the final ratings to retention
JL performance predictions# follow the instructions starting on
Page 38.
The flow chart on the next page shows the decision
algorithm that is built into the series of 10 questions. As
you go through the questions, you will note that the
instructions contain specific guidance with respect to the
-, decision points shown in the chart. It is important that
these instructions be followed carefully.
'08
[
II
U-
-
I
1Inu t e nd e to b e
Ised
IlE xcellent is theOuality of the
. ~Other
"iII) 3 �One Stop Mwy Steps
A
Other , d
, I
Re Surtn i- o
P.*qSF~~Rae Task Mental .. ."" T k MentalI
I
"~~"r Noe
o• r I
(• -I" No M.ate FaMts,
,M Must, be,,
<Other
STOPI
Question 1. Are job or memory aids used by the soldier in
performing (and in the performance evaluation of) this task?
answer Choice Scale Value
e Yes 1
* No 0
-[ Definitions
Job and memory aids are designed to guide or facilitate
Iithe soldier in on-the-job performance and to minimize the
i need for recall. Examples include:
,s Memory joggers learned in school, such asS - A - L - U - T - E
* Technical manuals or pamphletswhen used on the job to help do the task properly.
* Labels or instructions that are printed on or[ attached to equipment or containers.
* Checklists, flowcharts, worksheets, decisiontables, and system-fault tables.
* Manuals published by manufacturers to be used whileperforming maintenance tasks on equipment.
I * Instructions on forms that tell how to completethem. The form itself Is not a job aid, eventhough it has headings that indicate what shouldbe entered in the form. It Is considered to bean integral part of the task itself.
f. The key to accurately answering this question lies in
the way the task is intended to be evaluated and performed.
[1 (
For example, it is intended that a Technical Manual be used
while performing most maintenance tasks. That is the way
[ these tasks are taught and the way they are evaluated. If a
job or memory aid is not used while performing a task, and
none is used in evaluating that task, then the answer to
this question would be No.
All reference-dependent tasks; that is, tasks for which
required references are noted in the conditions section of
the task summary; should be considered job aided.
Tools or equipment needed to perform a task are not job
aids. However, if a supervisor is permitted to aid a
i soldier during the task performance evaluation, the
supervisor should be considered as a job aid.
I Write the Scale Value ("1" or *O") for the answer you
select in the first column, labeled "Job/Memory Aid," of the
Rating Form.
If you selected a No answer ("0") to this question,
I skip the next question and go to Question 3.
I If you selected a Yes answer ("10) to this question,
answer Question 2 on the next page. You will also be
J reminded of the fact that a job or memory aid is used to
perform this task on several of the other questions. Their
Srating will depend on how much support is provided by the
aid, which is covered by the next question.
U
I
NOTE: IF THERE IS NO JOB OR MEMORY AID,
DO NOT ANSWER THIS QUESTION.
GO TO QUESTION 3
Question 2. How would you rate the quality of the job or
memory aid?
Answer Choice Scale Value
0 Excellent. Using the job/memory aid, 56a typical soldier can do the entire taskcorrectly with no additional informationor help.
0 Very Good. With the job/memory aid, a 25typical soldier would need only a littleadditional information to complete thetask.
* Marginally Good. Even with the job/ 2memory aid, a typical soldier would needimportant additional information tocomplete the task.
* Poor. Even with the job/memory aid, 1a typical soldier would need a greatdeal of additional information inordez to complete the task.
Definitions
C This question requires you to think about the ability
of the job or memory aid to actually lead the soldier
C through the entire task without error.
There are several dimensions that help to define the
quality of a job or memory aid.
* Clarity - an excellent job aid presents the
[ information a soldier needs to perform the task in
a way that the soldier can understand; that is,the language and terminology matches the soldiers'level of understanding and reading ability. Ifnecessary, pictures, diagrams, tables and chartsare used to present critical information.
• Completeness - an excellent job aid provides allof the information the soldier needs to do the task.A complete job aid tells the soldier what,when, and how to perform at a necessary level"of detail. However, an aid that covers only aportion of a task very well is still less thanexcellent if other portions are left uncovered orare covered poorly.
0 Useability - an excellent job aid is useable"while the job is actually being performed. Forexample, a detailed, well-written technical manualis still less than an excellent job aid if thesoldiers cannot take the time to read itor if it is physically impossibleto use the aid while performing the task. In thesame way, a job aid may be excellent under someconditions but poor under others (e.g., darkness).
SIn some cases a task may have more than one job aid.
The rater should consider the overall excellence of the job
aids in relation to the entire task. Only if the job aids,
taken together, provide clear, complete and useful coverage
of the whole task can the rating on this question be
[ excellent.
[ The following examples for the task "Turn On Electrical
Test Panel" may help In making your choice:
Ezcellent job aid - Easy-to-read instructions printedclearly on the electrical test panel itself, tellingyou when to do the task, what to do, how to do it, andin what order to do it. Pictures are used to helplocate things.
[ Very Good job aid - A booklet that tells you basicallythe same information but it does not show where theknobs and switches are located. Operator errors aremore likely.
[ 73
* tiarginally Good job aid - Printed technicalinstructions that contain other information about thetest system mixed in with the needed information. Nopictures or diagrams.
Poor job aid - Technical reference manuals in whichgeneral principles of operation are given using complexlanguage - you must try to determine the actualprocedure for yourself.
Choose your answer using the above guidance and enter
the Scale Value for that answer on the Rating Form under the
column labeled 2, "Job/Memory Aid Quality."
If you select excellent as the answer to this question
I you will skip the next 3 questions. Look at these 3
questions now to see if your assessment of the job aid is
accurate. In effect, if a job aid is excellent it is the
same as saying the task has only 1 step (read job aid), no
particular sequence to remember (job aid tells you) and has
lots of built in feedback (job aid tells you if you are
doing each step correctly). If you have any doubts about
I- whether the job aid that you have rated excellent meets
[ these standards, you should down-grade your rating to very
good and answer the next 3 questions.
1. . .... .... .... . . .2. m m m • -
NOTE: IF YOU RATED TdE JOB AID AS "EXCELLENT"
6j DO NOT ANSWER THIS QUESTION OR THE NEXT TWO QUESTIONS.
GO TO QUESTION 6
Question 3. Into how many steps has the task been divided?
Answer Choice Scale Value
e One step 25
0 Two to five steps 14
e Six to ten steps 12
e More than ten steps 0
DefinitionsITFor purposes of this rating you should use the number
of evaluated performance measures listed in the task summary
[ under the Evaluation Guide as the number of steps. Where
there are non-evaluated sub-steps, these should also be
[. counted. However, do not include performance measures
relating to whether the soldier performed the task steps in
U sequence or within a certain time period. These are viewed
ponly as "scoring steps" and not as "task steps" for this
question.
U If the reference material does not provide sufficient
information, or if you feel that a task has not been
U7
ILaccurately divided into performance steps, the following
guidance may be helpful:
* A step is a separate physical or mental activitywithin a task, which has a well-defined, observablebeginning and ending point. It must be performedto complete a task correctly. Thus, "Ident~fying aTank" is one step, even though a number of mental
operations are needed to arrive at the correctanswer (e.g., note location of turret, countnumber of road wheels, etc.). These operations,however, are not observable and are not scored asseparate steps.
0 Steps should include all safety-relatedactivities, even though they are not directlypart of the task. The step "Check Backblast Area"is an observable step and is graded when measuringproficiency on preparing the LAW for firing.
o Tasks involving assembling or disassembling apiece of equipment tend to be multi-step tasks."Assembling the M16 Rifle" would be an exampleof a multi-step task and it is scored as such.
, A step in a higher Skill Level task (E-5 or above)may be a separate task at a lower skill level.This is as it should be, since one must assumethat the procedure was learned earlier and isno longer a separate task. For example,"Identify Terrain Features" is assumed athigher skill levels and is simply one stepin a task such as "Navigate with a Map." AtSkill Level One it is a separate task with manysteps of its own.
T Note that this question is skipped if the job aid for
E. this task was judged to be excellent. This is consistant
with the definition of an excellent job aid as one that
provides the soldier with complete information about each of
the steps of the task. Obviously, the number of steps in a
[ task would be irrelevant to the soldier who had forgotten
F them if the job aid presented them clearly and completely.
70
If steps are repeated in a task, the instruction to
repeat them should be counted as a step, but the repeated
steps themselves should not be recounted. For example, when
shifting indirect fire the soldier would repeat the steps of
estimating range, and communicating information to the gun
crew several times until the target was destroyed. Each
* repetition would not constitute a separate step.
If it is agreed that there are clearly more than 10
steps to a task, there is no point in trying to resolve
differences of opinion about the actual number. Anything
over 10 is given a 0 Scale Value regardless of the actual
number.
Determine your answer. Enter the Scale Value for this
answer on the Rating Form in the third column labeled
"Number of Steps."
[ If you select the first answer (One step), skip to
L[ Question 6.
C
SIa I |
NOTE: IF THE TASK HAS ONLY ONE STEP,
GO TO QUESTION 6
11Question 4. Are the steps in the task required to be
performed in a definite sequence?
Answer Choice Scale Value
• None are 10
e All are 5
.1 • Some are and some are not 0
Definitions
Some tasks are composed of steps that can be performed
in any sequence. For example, "Identify Terrain Features on
[La Map" is a task that is not scored for sequence in the
evaluation guide of the task summary. Such tasks should be
given a Scale Value of "10".
Other tasks, such as *Splint a Fracture," are made up
of steps that have only one correct sequence, Failure to1follow the particular sequence results in a "NO GO" on that
task. These tasks should be given a Scale Value of "5".
II A task that is a mixture of sequencad and non-sequenced
[steps should be given a Scale Value of "0". "Perform
Operator Maintenance on an M16A1 Rifle" is such a task.
U Only steps 1-5 nre scored for sequence.
r7
The reasoning behind this question is that it is easier
to remember to do a task when sequence does not matter.
[i However, if sequence is scored, it is easier to remember a
specific sequence for all steps than for only some of the
[ steps.
If a task, or parts of it, are supposed to be performed
in sequence, there must be a statement to that effect in the
I task summary; e.g., "Do, in order, all steps to clear the
object from the casualty's throat." In the absence of any
statement about sequence, it should be assumed that sequence
1 for that task is not scored, even though there may be a
natural or preferred order to doing the steps.
Choose your answer. Enter the Scale Value for your
answer in the fourth column (labeled "Sequence") of the
Rating Form.
7 C
Ii:
I
NOTE: IF THE VASK HAS ONLY 1 STEC
SKIP THIS QUESTION AND GO TO
QUESTION 6
Question S. Does the task provide built-in feedback so that
you can tell if you are doing each step correctly?T
Answer Choice Scale Value
* Has built-in feedback for all steps 22
• * Has built-in feedback for most steps 19
(50% and above)
ili* Has built-in feedback for only a few steps 11
(up to 50%)
a Has no built-in feedback 0
I. Definitions
[ Examples of tasks that provide built-in feedback are:
1• Disassembling a piece of equipment in whichremoving one section automatically uncovers thenext section (e.g., opening up a container to
U remove contents).
* Equipment operation in which the steps form alogical or natural progression. For example,
F. radio operators are expected to adjust or turn onseveral dozen switches in a certain orderwhen "powering up" their radio sets.However, the equipment is arranged so thatthe operator can follow a natural right-to-leftor left-to-right progression.
Ve
0 Assembling a subpart that does not fit the larger- assembly, thus indicating that some earlier step
was incorrect.
I Any task where there is some observable effectL due to the soldiers' actions, e.g., warning light,
- buzzer, meter reading, and the like.
For some tasks, the completion of the task provides an
I" automatic check on the correctness of it. For example,
'Changing a Tire" wouli have some of those characteristics
I (e.g., parts left over, wheel does not turn). The impact of
an artillery round would have a similar effect. However,
such end-of-task feedback may not assist the soldier in
Ji performing the steps of the task correctly in the first
place, and should not be considered in answering this
j; question.
The important point to consider in sý.Lecting an answer
to this question is whether the feedback to the soldier
J: indicates the correctness of his performance at each step.
Feedback that simply indicates that the step was completed
is not the kind of feedback that this question is
[ addressing.
Steps that have the least built-in feedback tend to
have many branching routines ("If A, then B"), or have
E safety checks that break the flow of a task's steps (e.g.,
OPlace the selector on SAFE before cleaning the rifle").
, Do not confuse the feedback that a performance test
administrator may give to a soldier to allow him to continue
F S I
the test after an error is made, with the feedback that is
being addressed by this question. We are concerned only
"with feedback that is an inherent part of the task, not an
artifact introduced by test conditions.
Before answering this question you may wish to look
back to question 3 to see bow many steps you identified for
the task. You need to consider each of them in arriving at
the correct answer to this question.
ji Enter the Scale Value for your answer on the Rating
Form in the column headed "Feedback.*
I
I
r
Question 6. Does the task or part of the task have a time
- limit for its completion?
Answer Choice Scale Value
* There is no time limit 40
i s There is a time limit, but it is fairly 35
easy to meet under test conditions
* There is a time limit and it In difficult 0
to meet under test conditions
Definitions
Ii The first choice means that no time limit has been
ji established for the task or any part of the task, so that a
"GO" may be achieved even though one soldier may take much
1 longer to do the task than another soldier. This choice is
also appropriate when a time limit is so liberal that no one
Ii ever fails to meet it.
[ The second choice above applies to those tasks, such as
wAssemble the M60 Machinegun," that have a time limit that
some soldiers may find difficult to meet. In this case, the
task summary has set a time limit that "pressures' the
average soldier a bit, but only a few would get a "NO GO"
because of it.
The third choice is for tasks that have a time limit
that is difficult to meet. Safety and combat-related tasksa
Isuch as "Sight a Target Through the Gunner's Telescope"
within 10 seconds would fall into this category. Soldiers
L being tested on this kind of task often get a 'NO GO" on the
basis of time alone.
Time limits, if any, are indicated in the task summary
j - in the standards section or as the last item in the list of
performance steps to which they apply. Some examples are:
1 'Put On, Clear and Check Mask" within 9 seconds, or
"Complete Steps 1 Through 5 in 9 Seconds or Less."
(Remember, however, that the time limit statement itself is
-__ not counted as a step.) If no time limit statement is found
in the task summary it may be assumed that there is none.I:
I:
I:
Question 7. How difficult are the mental processing
requirements of this task?
Answer Choice Scale Value
9 Almost no mental processing requirements 37
* Simple mental processing requirements 28
* Complex mental processing requirements. 3
- Very complex mental processing requirements 0
Definitions
This question usually cannot be answered entirely on
the basis of the task summary (as could the time and
sequence questions) but often must be deduced from a careful
reading of the summary and first-hand knowledge of the task
itself. This question gets at the difficulty of the thought
[• processes that a soldier must carry out during task
performance. Such processes as often described by terms
£ such as "thinking," "reasoning," "analyzing," "judging,"
*inferring", and =problem solving".
Be careful not to confuse this question with the next 2
IIquestions which deal with the number of facts, terms, etc.#
that must be memorized and the difficulty of those facts.
terms, etc., to memorize. Here we are concerned only with
rSIi
* .bat you have to do with the recalled information in order
to perform the task currectly.
A task requires almost -o mental processing if it is
essentially physical, or highly repetitive (e.g.r *Marching
in Line," "Salutingu).
I A task requires simple mental processing if it involves
making gross comparisons; estimating relative size, weight,
or distance; or performing simple computations.
SComplex mental processes require the soldier to make a
[• choice or decision based on subtle but discrete clues (e.g.,
prioritizing fixed targets, identifying different types of
1*aircraft or vehicles).
A task requires very complex mental processes if it
requires rapid decisions based on detailed, technical
I information, often under stress (e.g., planning an attac.k,
troubleshooting complex equipment).
In answering this question, consider the impact of a
ii job or memory aid on the thinking requirements of this task.
However, note that job aids are generally less helpful in
the area of higher thought processes than they are in the
IIareas of rote memory or proceduralized (step-following)
tasks. Nevertheless, an excellent aid may reduce a very
complex mental processing task to a complex mental
processing task.
C So
Question S. Now many facts, terms, names, rules or ideas
must'a soldier memorize in order to do the task?
Ahnswer Choice Scale Value
[ e None (or the job/memory aid 20
provides all necessary information)
e A few (1 - 3) 18
. Some (4 - 8) 13
. Very many (more than 8) 0
"1. Definitions
This question gets at the number of isolated pieces of
Vi information a soldier must remember in order to do the task
(not how hard it is to remember them, which is asked in the
next question).
Examples of the types of information that may have to
[ be remembered are:
JT Military nomenclature (terms)
* Conversion formulas
* Codes or call numbers
9 Technical names, specifications or tolerances
e *Doctrinal principles or rules of thumb
[
* Remember to consider the impact of the job or memory
aid (if any) in answering this question. If there are
* facts, terms, etc.# that are needed in order to do the task,
but some or all are covered in the job aid, your answerp
* should reflect this.
* The steps required to perform the task should not be
considered in answering this question.- It is the facts,
terms, etc., that must be remembered to do these steps that
are being addressed by this question, not the steps
V themselves.
This question (and the next one) do require some
judgment about the level of experience of the typical
soldier doing the task being rated. One could safely assume
that the typical soldier who has completed AZT knows basic
military terminology and concepts, and that these should not
[be included-in deciding on the answer to this question.
Those facts, terms, etc., unique to the task being rated,
I: however, should be included.
SHere again, if.the number of facts, etc., is obviously
greater than 8, there is no need to seek consensus on this
Iiquestion - it will get a Scale Value of "0" regardless of
jTthe outcome.
Select your answer and then enter the Scale Value for
that answer in the column labeled "Number of Facts."
8-- | I l l | I I 8
Question 9. How hard are the facts, terms, that must be
remembered?
uAinswer Choice Scale Value
"o Not applicable - there are none to 34remember or the job or memory aid
i! provides all of the needed information
0 Not hard at all - the information is 31simple
* Somewhat hard - some of the information 12is complex
0 Very hard - the facts, rules, terms, 0etc., are technical or specific to thetask and must be remembered in exactdetail
Definitions
This question rates the difficulty of the facts, terms,
etc., needed to do the task (even if there are only a few).
Facts and terms that have a close connection to the
[ task itself are more likely to be remembered. For example,
the terms *firing pin," and "whip antenna' have a logical
relationship to their function and are easy to recall.
Specific, detailed, or technical information that is
unrelated to the task is more difficult to recall. Call
iisigns and radio frequencies are examples of
difficult-to-recall information since they are purposely
.i assigned at random but must be used with precision. Also,
[ unorganized facts and terms (e.g., much military
[t
do
.= nomenclature) are more likely to be forgotten than facts and
terms that are part of a system (e.g., the phonetic
w alphabet).
The amount of help provided by job and memory aids
applies very directly to this question. They are often
[ designed specifically to help the soldier recall information
that is quickly forgotten (e.g.v 6-A-L-U-T-E).
Choose your answer and then enter the Scale Value for
your answer in the column labeled OHow Hard to Remember."
V
[I
12Ii
Question 10. What are the motor control demands of the
- task?
Answer Choice Scale Value
0 None 2
* Small but noticeable degree of 0motor control required
S• Considerable degree of motor 16control needed
0 Very large degree of motor 3control needed
Definitions
This question has to do with the level of precision
-" and/or accuracy of finger, hand and arm movements, not with
large body movements. Thus a task would be given a nor.. if
it involves only sheer physical strength or simple,
reflexive actions (e.g., pushing, lifting, carrying).
A small but noticeable degree of accuracy/precision is
C required by tasks such as driving a nail or adjusting a
r carburetor screw.
A considerable degree of motor control is needed for
0 tasks such as typing, driving a manual transmission car or
I' tracking a moving target.
A task requiring a very large degree of demands would
U be the repair of a very delicate piece of equipment, such as
r a microcircuit chip, or sending Morse code using a key.
S• 1
LSome tasks may combine both a strength component and a
S" motor control component. ?or example, a fairly heavy piece
of equipment may have to be positioned in a precise
location. In such cases, a value of considerable or even
t very large would be appropriate, depending on the degree of
motor control required.
Almost all tasks require some speaking skills and many
of them require written skills as well. While these are
indeed complex motor demands, they are considered to be
I already in the repertory of the typical soldier, and
therefore should not be included in making this rating.
However, typing or sending Morse code may be an integral and
U-unique part of a task. Therefore, they should be considered
in selecting a Scale Value for this question.
While the Scale Values associated with this question
[ may appear to be incorrect, since they are not in decending
order as are all the others, they are, in fact, the correct
Svalues. A task with a mall degree of motor control proves
to be more difficult to remember than a task with a
Cconsi derable degree of motor control. This finding is
reflected in the low value for the former (0u) and the high
value for the latter (016).
II Select your answer and enter the Scale Value in the
column labeled "Motor Control Requirements. 3
U
1II. Calculating a Total Score
This completes the instructions on the rating procedure
Itself. Check back to make sure you answered all the
questions that apply to the task being rated. The questions
you skipped (if any) should be blank on the Rating Form.
All others should have a 0 or a number written in. If you
rated the first question a "1" there should be a rating for
the second question; if you rated the first question a "0",
the second question should be blank. If question two has a
rating of "56" the next 3 questions should be blank.
Remember, if you change a rating for one question then
you must also change all the other questions that would be
effected by that change and adjust your Total Score
accordingly. (See the flow diagram on page 12 for a qraphic
[ representation of how the questions interact.)
The Total Score should be the one that reflects the
[ input of all team members. When this process has been
completed, a new Rating Form should be prepared and dated
that is labeled "Approved Task Rating Form." That form will
IIbe the one kept for future reference. Subsequent changes in
the way a task is performed as reflected in the task
jsumary, will probably require that its rating be revised.
At that point a new form should be prepared and dated
C reflecting those changes.
[
E 93•
IV. Using the Performance Prediction Tables
The two Performance Prediction Tables (pp. 41 and 42)'
provide the performance predictions for tasks that have been
rated. The numbers within the body of the table represent
the expected proportion of soldiers in a i'nit able to
perform a task correctly after up to 1 yea. of no practice
since a task was last performed correctly.
The first Table gives these predictions at monthly
intervals, up to 12 months. The second Table gives these
estimates at weekly intervals, up to 26 weeks.
To find a specific task retention prediction, first
I. locate the score in the left column that corresponds closest
to the obtained Total Score. Then read across either Table
until you reach the time interval you are concerned about
I: (shown at the top of the Table). The Table entry at that
point will be the proportion of soldiers that could be
expected to perform the task correctly at that time
[ interval.
For example, the Total Score from the Rating Form on
Task X is "140.w You want to know what percentage of
F soldiers can still perform the task 4 months after they last
practiced it. The entry in the first table under "4"
[ (months) is 36 percent. The entry in the weekly table is
r also 36 percent.
['
A second way of using the tables is as follows: If a
task has a score of 0140," how often should sustainment
training be provided in order to have at least 50 percent of
the soldiers proficient on that task at all times? Looking
at the line on the table next to the value "140,"-we see
that 46 percent is in the column headed w3" (months). This,
then, is the training frequency required to sustain a level
of proficiency on that task of approximately 50 percent.
A third way to use the tables is to determine what
level of proficiency you would expect in your unit if you
could only provide sustainment training every "X" months or
weeks. By looking at the Total Score for each of your tasks
[you can see what the percentage is for each one under the
appropriate months or weeks column. For those tasks with a
Ji rating of "130" for example, the percentage of proficiency
r at a 4-month frequency of training schedule is 25 percent;
for tasks with a rating of "170" it is 81 percent, and so
[ on.
[A I95[!
I!;
1 -Z l l I I I I
ILL
[
141
96•
S.... . . -L'nnmm u nnl IIlIIIIm
II
[
C 4
o 2
o
c
C3 Id*
8 2 Z
[ IS9 7
* oi i l S
R~~~~~~~ 0 0•8 a•R8•m "• ' a 0U *.
* -
; o
•~~~~~~~a a a 2 nIIH
asSaS
~q
li
op
Ii 1.a s
U p98
APPENDIX C: Questionnaire Administration Protocol
99
1. i'm Chavis Patterson
2. I'm here from ARI, the Army ResearchInstitute.
3. TRADOC has asked us to help them improve 63Wtraining. They have some concern about howwell 63W are doing their job.
4. The approach we're taking today is to find outif the 63W are forgetting some importantmaterial before they get a chance to practiceit in the field.
We know the time it takes to forget differentkinds of Army tasks but we need SMEs to tellwhat kinds of tasks these are.
5. That's where you come in Mr.
10. ARI developed this 10-item questionnaire forfield commanders so that they can find out howoften they should train their people on eachof their unit's tasks.
11. Lets look at the questionnaire rating form
and see what it involves.
12. The answer sheet has the 10 items right on it.
13. Here's the explanation for each of thecolumns.
14. Now let's look at the tasks we're going torate. These task lists are taken directlyfrom the organizational manuals that the 63Wshave available.
15. The task name is in the center, and morespecific info. is in the upper right corner.The steps for the task are listed down theleft side. There is an arrow in the 4thcolumn to show the steps that are done insequence. The 5th column shows what theresult should be, and the last two columnsshow the safety and tools required.
16. Let's look at the first task. Are you familiarwith this task or should we look at anotherone? Is this the way the task is done orshould be done? Any comments?
100
17. Now let's go to the questionnaire.
END OF ADMINISTRATION:
1.We have gotten your unbiased opinion. Next wehave to get you together with the other SMEsto get a consensus opinion from all of you. Can we meet againat ?
l1 0