DOCUMENT RESUME

ED 398 371 CE 070 975

TITLE Instructional Strategies [in HRD].PUB DATE Mar 95NOTE 35p.; In: Academy of Human Resource Development

(AHRD) Conference Proceedings (St. Louis, MO, March2-5, 1995); see CE 070 958. For other conferencesymposia, see CE 070 959-981.

PUB TYPE Speeches/Conference Papers (150)

EDRS PRICE MF01/PCO2 Plus Postage.DESCRIPTORS Adult Education; Employee Attitudes; Employer

Attitudes; Employer Employee Relationship;*Employment Practices; Ethics; Human Resources;*Labor Force Development; *Management Games; *On theJob Training; *Outcomes of Education; TeachingMethods; Work Attitudes

ABSTRACTThese four papers are from a symposium on

instructional strategies that was facilitated by Joseph Kessels atthe 1995 Academy of Human Resource Development (HRD) conference."Developing Supervisory Expertise: The Effects of an Inductive versusa Deductive Training Method on the Job Behaviors of Supervisors"(Margaret C. Lohman) discusses a study that showed thatsupervisors in a group trained w-L01-dedlictiVe methods demonstrated agreater ability Lo apply their learning, supervisors trainedinductively perceived that the quality of their training experiencewas better; implications of these findings for HRD theory, research,and practice are presented. "Identifying Instructional Criteria inCorporate Settings" (Clark J. Hickman) reports on a factor analysisstudy of a national sample of 372 trainers who reported the criteriathey employ when deciding to adopt or reject a proposed newinstructional method. "Exploring the Use of Training Games" (James J.Kirk, Hal Shoemaker) explores differences in the use of games bytrainers in 82 companies. Trainers who spent a large percentage oftheir training time on gaming activities used games to generate ideasor solutions, whereas trainers using games less tended to use them tointroduce new concepts. "Structured On-the-Job Training: Domain andFactors" (Jong Cheul Yang) critiques previous definitions ofstructured on-the-job training and suggests a model for the process.(KC)

***********************************************************************

Reproductions supplied by EDRS are the best that can be madefrom the original document.

***********************************************************************

Minor changes have been made to

originating it.

improve reproduction quality.

received from the person or organization

"PERMISSIONMATERIAL

TO REPRODUCE THIS

44240/

HAS BEEN GRANTED BYED°fATIONAL RESOURCES INFORMATIONf Educational Research and Improvement

CENTER (ERIC)

EPARTMENT OF EDUCATION

This document has been reproduced as

o Points of view or opinions stated in thisdocument do not necessarily representofficial OERI position or policy.

TO THE EDUCATIONAL RESOURCES

INFORMATION CENTER (ERIC)."

ACADEMY OFHUMAN RESOURCE DEVELOPMENT

1995 Conference Proceedings

March 2-5, 1995St. Louis, MO

[181 INSTRUCTIONAL STRATEGIES Facilitator: Joseph Kessels, University of Leiden (The Netherlands)

Developing Supervisory Expertise: The Effect of An Inductive Versus DeductiveTraining Method on the Job Behaviors of Supervisors

s Margaret C. Lohman, University of IowaIdentifying Instructional Criteria in Corporate Settings

Clark J. Hickman, University of Missouri- St. LouisExploring the Use of Training Games

James J. Kirk, Western Carolina UniversityHarold L. Shoemaker, University of Tennessee

Structured On-The-Job Training: Domains and FactorsJong Cheul Yang, University of Minnesota

Esir COPY AVAINABILE

2

Developing Supervisory Expertise: The Effects of an Inductive Versusa Deductive Training Method on the Job Behaviors of Supervisors

Margaret C. LohmanUniversity of Iowa

This study investigated the effects of an inductive versus a deductive trainingmethod on the learning outcomes and job behaviors of supervisors. Thefindings showed that while supervisors in the deductive group demonstratedgreater ability to apply their learning, supervisors in the inductive groupperceived that the quality of their training experience was better.Implications of these findings for HRD theory, research, and practice arediscussed.

Job expectations for supervisors have been changing in recent years. In the past, supervisors wereexpected to use a directive approach to supervise their employees (Bittel, 1987). With this type ofapproach, supervisors directed the work of subordinates by controlling their behaviors and actions(Bienvenu, 1976). In general, this method of supervision was consistent with the goals of mostorganizations, which were oriented toward high production and efficiency. However, during recentyears many organizations have responded to the demands of a new economy by shifting their goalorientations from production and efficiency to quality and customer responsiveness. An outcomeof these shifting goal orientations has been a growing recognition that employee knowledge andskills are critical to the effectiveness of organizations (Kirkpatrick, 1993). As a result, a directivesupervisory approach, which tends to control employee behavior rather than develop employeecapacity, has become increasingly less effective as a means of managing work (Bramlette, 1984;Wolfe, 1983).

Presently, organizations expect supervisors to facilitate the work of their employees(Bittel, 1987). In contrast to a directive approach, a facilitative supervisory approach involvesfostering a work environment that values the development of job expertise, knowledge-seekingability, and learning skills. According to Wolfe (1983) and Bramlette (1984), a prime advantage ofa facilitative supervisory approach is that it fosters a work environment which betteraccommodates the new and dynamic nature of work and the changing nature of those who do thework.

Present supervisory job expectations have focused greater attention on how supervisorsare developed, especially as the development involves formal training programs (Bittel, 1987;Gardner, 1980). Many organizations provide HRD programs for preparing supervisors. Previousresearch suggests that outcomes of these programs are determined by both the content of theinstruction and the methods used to deliver the content (Burke & Day, 1986). The appropriatenessof the training method may be especially important given that supervisory training programs arepresently emphasizing more problem solving, critical thinking, and learning skills.

Traditionally, supervisory training programs have used deductive training methods. Thesemethods are characterized by off-site training sessions in which instructor-led presentations ofcontent are followed-up by group discussions and small-group activities (Holland, Holyoak,Nisbett, & Thagard, 1986). A review of the related literature in HRD, educational psychology, andcognitive psychology suggests that there are three primary concerns regarding the use of deductivemethods for supervisory training. First, training methods which are more deductive in nature maynot necessarily be aligned with actual supervisory job expectations. Although deductive methodsare commonly associated with knowledge and skill acquisition and, to a somewhat lesser degree,retention and transfer, one of their limitations is that they do not necessarily require the use of, orresult in the development of, higher-level cognitive skills (Newbert & Binko, 1992). Thissituation is concerning since present job expectations require that supervisors possess greaterproblem-solving, critical-thinking, and learning skills.

A second concern with the use of deductive methods for supervisory training is that thedevelopment of and ability to use complex cognitive skills may be diminished because of thelocation where training typically occurs (Gardner, 1987). Supervisory training which utilizesdeductive methods often takes place away from the job site. As a result, trainees are removed fromthe influence of people and environments that provide rich opportunities for acquiring, transferring,and applying information.

0 copyright, M. Lohman, 1995

1 8- I

3

A third concern is that the use of deductive methods for supervisory training may notaccount for the relationship between training method and training content. Studies have found thatcertain training methods are more suited for particular content areas (Burke & Day, 1986; Carroll,Paine & Ivancevich, 1972). Yet, supervisory training programs seem to rely on the continual useof deductive methods without considering whether other types of training methods may be bettersuited for specific content areas.

In contrast to a deductive method of training, guided discovery is an inductive method thatappears to be more consistent with present supervisory job expectations. Featuring a moreinductive sequence of instructional events, guided discovery involves trainees in three mainlearning tasks: (a) gathering, organizing, and categorizing information; (b) identifying criticalrelationships among information and making inferences based on those relationships; and (c)testing inferences by applying them to new situations (Joyce & Weil, 1986; Newbert & Binko,1992; Taba, 1966). Previous research has shown that inductive methods are more effective indeveloping higher-level cognitive skills, all the while accomplishing the same short-term trainingoutcomes as deductive methods (Newbert & Binko, 1992; Vazquez-Abad & Winer, 1992).However, these studies have been primarily conducted in academic and laboratory settings withchildren and college students. Therefore, while an inductive method appears to be more alignedwith present supervisory job expectations, the theoretical proposition that it effects thesimultaneous development of technical knowledge and cognitive skill has not been empiricallyexamined with adults involved in training activities.

Purpose of the Study

The purpose of this study was to determine the effects of a guided discovery versus a deductivetraining method on the learning outcomes and job behaviors of supervisors. As illustrated inFigure I, it was proposed that if a training method can affect learning outcomes and job behaviors,then it is likely that trainees receiving the same training content will demonstrate different learningoutcomes and job behaviors, because of the training method used to deliver the content.

Training

Method

Learning

Outcomes

Job

Behaviors

Figure 1. Theoretical Model of the Relationship Between Training Method. Learning Outcomes,and Job Behaviors

Hypotheses

This study investigated the effects of a guided discovery versus a deductive training method onsupervisors' acquisition of learning, application of learning, self-reported problem-solving skill,and facilitation of subordinate problem-solving skill. Specifically, four hypotheses were tested:

Acquisition of Learning. Supervisors experiencing the guided discovery methodwill acquire greater mastery of the concept taught during training than those experiencing thedeductive method.

Application of Learning. Supervisors experiencing the guided discovery methodwill demonstrate greater ability to apply the concept taught during training to a job task than thoseexperiencing the deductive method.

Self-Reported Problem-Solving Skill. Supervisors experiencing the guideddiscovery method will demonstrate greater levels of problem-solving skill when handling achallenging job task than those experiencing the deductive method.

BEST COPY AVAINA LE18-1

4

Facilitation of Subordinate Problem-Solving Skill.. Subordinates ofsupervisors experiencing the guided discovery method will report that their supervisors expectedthem to demonstrate greater levels of problem-solving skill when learning the concept taughtduring training than will the subordinates of supervisors experiencing the deductive method.

Methods

In the following paragraphs the subjects, research design, outcome measures, rating instruments,and data analysis plan that were used in the study are discussed.

Subjects. This study was conducted during the first week of March, 1994. Theexperimental treatment was administered in an existing supervisory training program offered by theHRD department of a large financial services organization. This company is located in the mid-west region of the United States and is large in size, employing 15,526 people at the time of thestudy. Supervisors represent 5.9 percent of this company's total employee population.

Although 19 supervisors originally enrolled in the supervisory training program, 3 didnot attend two mandatory training sessions and were considered drop-outs of the study. Theremaining 16 supervisors served as the subjects in the study.

Of these 16 supervisors, 10 were female and 6 were male. The subjects was fairly youngand well educated. The mean age was 33.5 years and 12 supervisors had attained a college degree orhigher. In addition, the majority of the subjects (n = 10) were new supervisors with less than 18months of supervisory experience. The supervisors were also fairly new to the organization with 9subjects having less than 5 years of service with the company, 5 subjects having 5 to 15 years ofservice, and the remaining having more than 15 years of service. The supervisors worked in avariety of functional areas with 5 supervising a production-oriented area, 5 supervising anaccounting/clerical area, 3 supervising a sales/marketing area, and 3 supervising anengineering/technical area.

Research Design. This was an experimental type of study utilizing the post-test onlycontrol group design for the main independent variable (Campbell & Stanley, 1963). Subjectswere randomly assigned to either a guided discovery or a deductive group for 1 of 11 instructionalunits in the training program. After the training was delivered, post-test measures were takenimmediately following training and three weeks after training. This design was chosen because itenabled the researcher to compare the guided discovery and the deductive method in terms of theireffects on the selected dependent variables. Furthermore, randomization assured that the groupswere equal on any confounding variables that might be related to the dependent variables, thusassuring that differences observed in the dependent variables could be attributed to the treatmentrather than to other causes (Kerlinger, 1964).

The guided discovery and deductive versions of the instructional unit were designed by theresearcher. The content of the instructional unit, the constructive feedback process, was derivedfrom an existing instructional unit used by the participating organization. A subject matter expertin the participating organization validated the content of both versions of the unit. In addition,two instructional design experts at The Ohio State University validated the guided discovery andthe deductive methods used in the different versions of the unit.

Two experienced trainers employed by the HRD staff of the participating organizationdelivered the training. Separate two-hour training sessions were held for each of these trainers.The training objectives, method, activities, and evaluation methods that trainers were expected touse when delivering the training were discussed during these sessions.

The trainers delivered the training during two off-site sessions, which were held sevendays apart. Although the total amount of instructional time was constant for both groups (4hours), the format of the sessions varied with the method of training. For the guided discoverygroup, the first session lasted one hour and included discussion of the training topic, the learningobjectives, and the on-the-job training (OJT) activity. For the OJT activity, supervisors identifiedpeople in the workplace who possessed expertise in giving constructive feedback and then met withthese individuals to collect information about the constructive feedback process. Trainees wereasked to complete this activity before the second training session. The second training sessionlasted three hours during which time supervisors engaged in three inductive learning activities.First, in small groups supervisors shared and categorized information that they had collected duringtheir OJT activity. Next, in a large group they explored relationships within and between their

18-1

categories of information and used their resulting knowledge to make inferences about the criticalsteps in the constructive feedback process. Lastly, supervisors tested and revised their inferencesby analyzing video-taped and live role-plays of constructive feedback situations.

In contrast to the guided discovery group, the first training session for the deductive grouplasted three hours. During this session, the trainer presented the topic, objectives, content oftraining, and instructions for an OJT activity. Supervisors then practiced applying the content bycritiquing video-taped and live role-plays of the constructive feedback process. At the end of thissession, supervisors were assigned an OJT activity to be completed before the second trainingsession. This activity involved practicing the constructive feedback process in a real job situation.The second training session lasted one hour and included a large group review of the constructivefeedback process and a discussion of the results of the Off activity.

Outcome Measures. Four dependent variables assessed the effects of training.Acquisition of learning was classified as a learning outcome and was measured immediately aftertraining. Application of learning, self-reported problem-solving skill, and facilitation ofsubordinate problem-solving skill were classified as job behaviors and were measured three weeksafter training. Each dependent variable was measured by using a rating instrument to evaluatewritten examples of the criterion. According to research and design experts, for example Kerlinger(1964), rating scales are an appropriate technique for quantitatively analyzing the content of writtenresponses.

Acquisition of learning was measured by evaluating written examples of hypotheticalconstructive feedback scenarios generated by supervisors. These examples were evaluated in termsof the degree to which they demonstrated six key steps of the constructive feedback process.

Application of learning was measured by evaluating written responses of supervisors inwhich they described how they had recently used constructive feedback in a job situation. Theseresponses were evaluated in terms of each supervisor's application of six key steps of theconstructive feedback process to a job situation.

Self-reported problem-solving skill was measured by evaluating the level of problem-solving skill that was evident in written responses of supervisors in which they described theirmental thoughts about how they identified and solved a problem and the physical actions that theyused to handle the problem. These responses were evaluated in terms of the degree to whichsupervisors gathered information about a problem situation, interpreted the meaning of theinformation in relation to the problem situation, and applied those interpretations to the situationto test their validity and usability.

Facilitation of subordinate problem-solving skill was measured by evaluating the level ofproblem-solving skill that supervisors expected their subordinates to demonstrate when engagingin a challenging task. The job task that was used as a referent was learning about constructivefeedback. At the end of the second training session, supervisors were instructed to pass on themeaning of constructive feedback to their subordinates using any methods that they deemedappropriate. Three weeks later, a sampling of each supervisor's subordinates reported how theirsupervisor had expected them to learn about constructive feedback. These responses were thenevaluated in terms of the degree to which subordinates were expected to gather information aboutthe constructive feedback process, infer from this information the key steps of the constructivefeedback process, and apply those inferences to test their validity and usability.

Rating Instruments. Two rating instruments were developed by the researcher tomeasure the dependent variables. The Concept Learning Rating Scale (CLRS) was designed tomeasure acquisition and application of learning. This instrument consisted of 6 items, whichrepresented the key steps of the constructive feedback process as identified in the instructional unit.A weight for each of the six items was established by a panel of constructive feedback experts. A4-point scale ranging from I (poor) to 4 (excellent) was used to measure the quality with which aresponse demonstrated each of the six steps. Scores for a response were obtained by assigning arating for each of the six items, multiplying each rating by its assigned weight, and summing theweighted scores. Total scores were reported as interval data with a range of 1 to 4.

The Problem Solving Rating Scale (PSRS) was designed to measure the two problem-solving skill variables. The instrument used a 4-point scale ranging from 0 (not evident) to 3

(much evidence) to evaluate the level of evidence that was present in a response for each of nineproblem-solving activities. A total score was obtained by summing the ratings assigned to each ofthe nine items. Total scores were reported as interval data with a range of 0 to 27.

18-1

6

The validity of both instruments was established. First, a panel of subject matter,population, and research experts established the face and content validity of the instruments. Next,two field tests, comprised of supervisors who had previously taken the supervisory trainingprogram in which the study was being conducted, confirmed the face and content validity of bothinstruments and established their suitability for supervisors in this particular organization.

After validity was established, a pilot test was used to assess the reliability of theinstruments. Supervisors attending a supervisory orientation program in February, 1994 served asthe pilot group (N = 10). Participants generated two written responses for the pilot test; in thefirst response they described how they had recently given constructive feedback to an employee andin the second they described how they had recently handled a challenging job situation. Fourtrained raters then used the CLRS and the PSRS, respectively, to evaluate the responses. Test-retest reliability, interrater reliability, and an internal measure of consistency were assessed for eachinstrument. As shown in Table 1, reliability coefficients were very high ranging from .78 to .93.

Table 1. Reliability Coefficients for the CLRS and PSRS

Instrument

Measure of Reliability

Test-Retest Interrater Internal Consistency

CLRSPSRS

.84

.83.93.78

.83

.79

Data Analysis. A three-stage plan was used to analyze the data. First, a correlationalanalysis was conducted to determine if any of nine attribute variables may have influenced theresults of the study. An attribute variable was to be used as a covariate in an analysis ofcovariance if it was found to be at least moderately correlated (r > .35) to both the method oftraining and any of the dependent variables (Fraenkel & Wallen, 1990). None of the nine attributevariables met those conditions. The second stage of data analysis involved testing the fourhypotheses. Since no covariates were found, t-tests were used as the statistic for these tests. Thelast stage of data analysis involved analyzing the attitudes of trainees toward the trainingexperience.

Results

Ratings of responses generated by 16 supervisors and a sampling of each supervisor's subordinatesprovided the data for testing the hypotheses. Table 2 presents the mean scores, standard deviations,and t values for each of the four dependent variables.

A significant difference was found for one of the four hypotheses. At an alpha level of.10, no significant difference was found between the mean scores of the two instructional groupsfor acquisition of learning, self-reported problem-solving skill, or facilitation of subordinateproblem-solving skill. However, a significant difference (p = .027) was found for application oflearning with the deductive group demonstrating greater ability to apply the constructive feedbackprocess than the guided discovery group.

In addition to the results for the four hypotheses, an attitude survey was used to gatherinformation about trainees' attitudes toward the training experience. The survey consisted of twosections. The first section assessed participants' attitudes toward the clarity of the trainingobjectives, the consistency of the training content and the objectives, the relevancy of the contentto their jobs, the usefulness of the training materials, and the expected ability to use theknowledge /skill gained from the training experience in their jobs. A 5-point scale ranging fromstrongly disagree (1) to strongly agree (5) was used to measure these items. The second sectionassessed participants' attitudes toward the quality of four training activities in the training programand the overall training experience. A 5-point scale ranging from poor (1) to excellent (5) wasused to measure these items.

18-1

T-tests revealed no significant differences between the attitudes of the subjects in the twogroups on any of the items in the first section. However, data revealed that there were significantdifferences between the two groups regarding their attitudes toward the quality of the trainingactivities and the overall training experience. As shown in Table 3, supervisors in the guideddiscovery group rated the quality of three of the four training activities and the overall trainingexperience significantly higher than participants in the deductive group.

Table 2. Means. Standard Deviations, and T Values for the Instructional Groups on SelectedDependent Variables

GuidedDiscoverya Deductivea

Dependent Variables Mean SD Mean SD

Acquisition of learning 2.71 .82 3.31 .61 1.67Application of learning 1.82 .52 2.55 .66 2.46*Self-reported problem-

solving skill13.60 6.14 16.60 5.76 1.01

Facilitation of subordinateproblem-solving skill

2.50 3.70 1.90 2.70 - .39

an = 8 for each group*g < .10

Table 3. Attitudes of Supervisors Toward Four Training Activities and the OverallTraining Experience

Training Activities

GuidedDiscoverya Deductivea

Mean SD Mean SD

Large group discussion 3.6 .52 3.0 .76 -1.93*Small group activity 3.6 .74 3.5 .93 -0.30Role-play activity 3.6 .52 3.0 .76 -1.93*On-the-job activity 3.6 .75 2.9 .84 -1.90*Overall training experience 3.9 .64 3.1 .35 -2.90*

an = 8 for each group.*n < .10

Discussion

Theory in educational and cognitive psychology postulates that inductive methods effect theconcurrent development of technical knowledge and cognitive skill. This study investigatedwhether this proposition applies to adults engaged in training activities. The findings revealed thatwhile the two methods of training did not differentially impact acquisition of learning or problem-solving behaviors, they did differentially influence trainees' ability to apply their learning and theirattitudes toward the training experience.

The findings for acquisition of learning appear consistent with previous research andtheory. Training experts believe that training objectives can be achieved through the use of either

18-1

8nverir R~ PV WAIN A 1111f.15°.

a deductive method of training (Goldstein & Sorcher, 1974; Kirkpatrick, 1993) or a guideddiscovery method of training (Newbert & Binko, 1992), if the methods are utilized andimplemented appropriately. Moreover, empirical studies conducted with deductive methods (forexample, see Anderson, 1971; Burnaska, 1976; Harrison, 1992) and with guided discovery (forexample, see Craig, 1956; Kittle, 1957; Worthen, 1968) have confirmed that training objectivescan be achieved through the use of either method.

Surprisingly, the findings did not confirm the hypotheses for supervisory problem-solving skill and facilitation of subordinate problem-solving skill. Two explanations for thesefindings seem most plausible. First, it is possible that trainees were not exposed to the guideddiscovery method for a sufficient period of time to influence their problem-solving behaviors.Joyce and Weil (1986) and Newbert and Binko (1992) suggest that while immediate improvementsin problem-solving skill are not often found with the use of one particular method of instruction,the continual and repeated use of an inductive method will in all likelihood promote thedevelopment of problem-solving skill.

A second explanation for these findings is that certain organizational factors may not havesupported the types of behaviors that individuals typically manifest during problem solving. Oneof these factors may have been the culture of the organization. Many supervisors in the studyreported that a norm of this organization is that they are expected to solve problems on their ownas quickly as possible. In all likelihood, this type of culture inhibits the use or development ofproblem-solving skill. Another organizational factor that may have influenced supervisor andsubordinate problem solving was the goal orientation of the organization. Primary job challengesreported by supervisors involved managing people, projects, and tasks to meet productivitystandards and/or target dates. Seemingly, this type of production goal orientation leaves littleopportunity for using or facilitating problem-solving skill in the workplace. A third factor thatmay have influenced the problem-solving behaviors of supervisors and subordinates was theorganization's incentive system. A review of the participant responses indicated that little to noincentive was provided by the organization to supervisors for developing their own problem-solving skill or facilitating its development in subordinates.

Surprisingly, while the findings showed that the two methods of training did notdifferentially impact acquisition of learning or problem-solving skill, they did show thatsupervisors in the deductive group were able to apply their learning to a greater degree than theguided discovery group. Three rationales may possibly explain these findings. First, differences inthe sequence and purpose of the instructional events used in the two training methods may haveinfluenced subjects' ability to apply their learning. In the deductive group, participants practicedthe constructive feedback process in a role-play exercise and, afterward, applied the constructivefeedback process to a job situation in an OJT activity. While trainees in the guided discoverygroup also participated in a role-play and an OJT activity, the sequence of the learning activitieswas reversed and the objective for both activities was concept formation rather than conceptapplication. In contrast, guided discovery group members collected information about theconstructive feedback process during the OTT activity, and then made and tested inferences aboutthe steps in the constructive feedback process during the role-play activity. A comparison of thesedifferences suggests that supervisors in the deductive group may have had greater opportunity topractice the constructive feedback process in its final form under conditions highly similar to actualjob situations and, as a result, were able to apply their learning to a greater degree.

A second explanation for these findings is that the amount of time allotted for the OJTactivity, six working days, may have differentially impacted the two groups. Unlike the deductivegroup, most supervisors in the guided discovery group had to schedule a meeting with anotherindividual to complete the OJT activity. The limited amount of time may have prevented subjectsin guided discovery group from thoroughly completing the activity and, as a consequence,diminished their ability to benefit fully from subsequent training activities and apply their learningto job situations.

A third explanation for these findings is that the nature of the method used to evaluatelearning may not have fully reflected all of the learning which occurred. The CLRS was used toevaluate responses in terms of their demonstration of six pre-defined constructive feedback steps.While these steps were presented to the deductive group, the nature of the guided discovery methodcalled for that group to induce the steps through their learning activities. A review of thetranscripts and materials from the training sessions found that the guided discovery group created afive-step process; they had combined the six key steps identified in the instructional unit into two

18-1

steps and had added three other steps. After comparing the instructional unit's constructivefeedback process and the one created by the guided discovery group, several constructive feedbackexperts commented that the latter process was actually more inclusive. Thus, it is possible thattrainees in the guided discovery group learned a great deal, but were unable to demonstrate thatlearning because the assessment tool too rigidly measured learning according to a pre-definedprocess.

Paradoxically, while the findings showed that participants in the deductive group wereable to apply their learning to a greater degree with no differences between the two groups onproblem-solving skill, participants in the guided discovery group perceived that the quality of thetraining activities and the overall training experience was better than those in the deductive group.Different levels of trainee participation, which are inherent in the two training methods, mayexplain these findings. While supervisors in the deductive group passively received informationduring their training experience, those in the guided discovery group actively participated in mostphases of their training experience. Learning outcomes that have been associated with high levelsof trainee participation include greater motivation to learn, greater interest in the content of thelesson, and increased intrinsic satisfaction in learning (Newbert & Binko, 1992; Taba, 1966).Accordingly, it is possible that trainees in the guided discovery group enjoyed the high level ofparticipation that they experienced during training and reflected this satisfaction in their attitudesurveys.

In summary, it was found that while the deductive method resulted in greater applicationof learning, guided discovery was the training method that supervisors preferred. Furthermore,anecdotal data indicated that certain aspects of the training experience and of the organizationmediated the ability of supervisors to participate in, learn from, and apply the skills acquired duringtraining. However, several limitations of the study should be considered when generalizing itsresults. The sample was small in size (N = 16) and contained a fairly homogeneous group ofsupervisors. In addition, the treatment was embedded within a week-long supervisory trainingprogram and, although attempts were made to avoid overlap of training content, it is possible thatthe findings were influenced by the interaction of the treatment with the other instructional units ofthe training program. Accordingly, the generalizations of this study seem most appropriate forthose populations which have predominantly new and inexperienced supervisors and those trainingprograms which focus on developing basic supervisory skills. It is recognized that replications ofthis study with other samples and under different conditions are required to strengthen the validityof its generalizations.

Implications. Although this study did not confirm that an inductive training methodeffects the concurrent development of technical knowledge and cognitive skill, there are fourimportant implications of the findings for theory, research, and practice in HRD. First, more mustbe known about guided discovery before it can effectively be used as a method of training. Whilethe findings show that guided discovery was the method that trainees preferred, the evidence alsoshows that the deductive method enabled trainees to apply their learning to a greater degree.Perhaps further study of guided discovery with larger sample sizes and variations in some aspectsof the method (for example, varying the amount of exposure to the training method and theamount of practice with the training content in its final form) would find significant differences inthe development of technical knowledge and cognitive skill, all the while maintaining the highlevels of trainee interest, motivation, and satisfaction that were found in this study.

Second, the findings indicate that supervisory training programs should cautiouslycontinue to use deductive methods for training new supervisors. However, some evidence in theself-directed learning literature suggests that the desire and ability of trainees to participate in andlearn from different methods of training may vary with their level of expertise (Grow, 1991; Pratt,1988). For example, Gerald Grow (1991) in his Staged Self-Directed Learning Model matches thedegree of directiveness of the trainer with the desire and ability of the trainee to be self-directingduring the learning process. Correspondingly, it is plausible that more experienced supervisorsmay respond differently to the guided discovery method since they have greater knowledge, skills,and experience to utilize during the learning experience. Further investigation of the relationshipbetween level of expertise and method of training is clearly warranted.

Third, the findings suggest that evaluation of training should include assessments ofchanges in job behaviors and job performance. Traditionally, HRD practitioners and researchershave relied on exit surveys and, to a lesser degree, measures of learning as the means for evaluatingtraining programs (Werner, O'Leary-Kelly, Baldwin, & Wexley, 1994). The concern with these

18-1 0

methods of evaluation is not with their usefulness but with their completeness. The findings ofthis study showed that the attitudes of trainees toward training and their levels of learningimmediately after training were not consistent with their subsequent job behaviors. If the resultsof this study had been solely based on the first two measures, the study's conclusions would havebeen markedly different. As this situation illustrates, the usefulness of different training methodswould be better understood if the impact of those methods on job behaviors and job performancewas evaluated.

A fourth implication of this study is that the influence of an evaluation method on theuse of divergent and innovative thinking during training should be considered when designingtraining programs. As reported, the constructive feedback process created by the guided discoverygroup was different than, and in some ways superior to, the process defined in the instructionalunit. Yet, the rating scale that was used to evaluate learning was based on the defined process. Iforganizations truly value creativity, innovation, and inductive thinking, it appears that moreflexible methods of evaluation should be selected for training activities which involve the use ofand/or result in the development of these types of higher-level cognitive skills.

Future Research. Four research questions appear central to deepening presentunderstandings about the usefulness of inductive methods for training in organizational settings.One of these questions is: How do different groups of employees with varying levels of expertiserespond to inductive training methods? Since this study focused on one group of employees, newsupervisors, further examination of inductive methods with other groups of employees anddiffering levels of job expertise would expand current knowledge regarding the usefulness ofinductive methods for training purposes.

A second important research question is: What is the role of certain aspects of theinductive method in terms of achieving training outcomes? Data in this study suggest that thelevel of trainee participation, the degree and type of practice with the training content, the amountof exposure to the training method, and the method of evaluation may influence the attainment oftraining outcomes. Further study of these variables is required to clarify their role in achievingspecified training outcomes.

A third research question is: What types of training outcomes can be realized by usinginductive training methods? Although numerous learning theorists have proposed that inductivemethods result in greater acquisition and transfer of technical and cognitive skill than deductivemethods, these propositions must be empirically investigated to examine their validity for adultsinvolved in training activities.

A last research question that emerges from this study is: How do organizational factorsinfluence the development and use of various types of higher-level cognitive skills? While therehas been a fair amount of empirical study investigating the role of organizational factors in thetransfer and development of technical knowledge and skill, much less is known about how thesefactors influence one's ability to develop and use different types of higher-level cognitive skills inthe workplace. Considering that employees increasingly require higher-level cognitive skills to dotheir jobs, further study of how organizational factors influence the development and use of theseskills is warranted.

References

Anderson, M.H. (1971). The clinical instructor training program: Development. operation,evaluation. (Final report). Los Angeles, CA: California University. (ERIC DocumentReproduction Service No. ED 064 490)

Bienvenu, B.J. (1976). Supervisor status and training...an approach to the changing needs insupervisory development. In B.B. Boyd (Ed.), Supervisory training: Approaches andmethods (pp. 11-16), American Society of Training and Development.

Bittel, L.R. (1987). The complete guide to supervisory training and development. Reading,MA: Addison-Wesley.

Bramlette, C.A. (1984, March). Free to change. Training and Development Journal, pp. 32-40.

Burke, M.J., & Day, R.R. (1986). A cumulative study of the effectiveness of managerialtraining. Journal of Applied Psychology, 21(2), 232-245.

EST COPY AVAIIA LE

Burnaska, R.F. (1976). The effects of behavior modeling training upon managers' behaviorsand employees' perceptions. Personnel Psychology, 22, 329-335.

Campbell, D.T. & Stanley, J.C. (1963). Experimental and quasi-experimental designs forresearch. Chicago: McNally.

Carroll, S.J., Paine, F.T., & Ivancevich, J.J. (1972). The relative effectiveness of trainingmethods -- Expert opinion and research. personnel Psychology, 21, 495-509.

Craig, R.C. (1956). Directed versus independent discovery of established relations. Journal ofEducational Psychology, LE, 223-234.

Fraenkel, J.R., & Wallen, N.E. (1990). How to design and evaluate research in education.New York: McGraw-Hill.

Gardner, J.E. (1980). Training the new supervisor. New York: AMACOM.Gardner, J.E. (1987). Choosing effective development programs. New York: Quorum Books.Goldstein, A.P., & Sorcher, M. (1974). Changing supervisor behavior. Elmsford, NY:

Pergamon Press.Grow, G.O. (1991). Teaching learners to be self-directed. Adult Education Quarterly, 4j(3),

125-149.Harrison, J.K. (1992). Individual and combined effects of behavior modeling and the cultural

assimilator in cross-cultural management training. Journal of Applied Psychology, //(6),952-962.

Holland, J.H., Holyoak, K.J., Nisbett, R.E., && Thagard, P.R. (1986). Induction: Processesof inference. learning, and discovery. Cambridge, MA: MIT Press.

Joyce, B., & Weil, M. (1986). Models of teaching. Englewood Cliffs, NJ: Prentice Hall.Kerlinger, F.N. (1964). Foundations of behavioral research. New York: Holt, Rinehart and

Winston.Kirkpatrick, D.L. (1993). How to train and develop supervisors. New York: American

Management Association.Kittle, J.E. (1957). An experimental study of the effect of external direction during learning on

transfer and retention of principles. Journal of Educational Psychology, 48, 391-405.Newbert, G.A., & Binko, J.B. (1992). Inductive reasoning in the secondary classroom.

Washington, D.C.: National Education Association.Pratt, D.D. (1988). Andragogy as a relational construct. Adult Education Ouarterly, 31(3),

160-181.Taba, H. (1966). Teaching strategies and cognitive functioning in elementary school children

(Cooperative Research Project No. 2404). San Francisco, CA: San Francisco StateCollege.

Vazquez-Abad, J., & Winer, L.R. (1992). Emerging trends in instructional interventions. InH.D. Stolovitch & E.J. Keeps (Eds.), Handbook of human performance technology (pp.672-687). San Fransisco: Jossey-Bass.

Werner, J. M., O'Leary-Kelly, A. M., Baldwin, T. T., & Wexley, K. N. (1994). Augmentingbehavior-modeling training: Testing the effects of pre- and post-training interventions.Human Resource Development Quarterly, 5(2), 169-183.

Wolfe, E.H. (1983, March). Supervisory development: The need for an integrated strategy.Training and Development Journal, pp. 28-31.

Worthen, B.R. (1968). A study of discovery and expository presentation: Implications forteaching. Journal of Teacher Education, la, 223-242.

18-1 12

Identifying Instructional Criteria in Corporate Settings

Clark J. Hickman, Ed.D.University ofMissouri-St. Louis

A factor analysis study on responses from a national sample of 372trainers who completed the Trainer Method Acceptance Scale providedempirical evidence of six conceptually distinct factors associated with thecriteria corporate trainers employ when deciding to adopt or reject aproposed new instructional method. The results of this study suggest thatthose charged with trainer development may increase the likelihood oftraining-transfer ifthe rationale for changing instructional behcrvior alignswith criteria that trainers themselves consider important in making thesedecisions.

According to McCullough (1987), most corporate trainers were themselves not trained to teachother people, either in an educational environment in general or a corporate classroom inparticular. Moreover, corporate trainers naturally vary in training competence regardless of theirformal preparation. McCullough, reporting for the American Society of Training andDevelopment (ASTD), found that 98% of corporate trainers have at least one college degree;however, the rapid proliferation of technologies and teaching methods available to trainersmakes it imperative that they continue their professional development. Another reason for theimportance of professional development is the background of most trainers. In the surveyconducted by ASTD (McCollough, 1987), only 24% of the trainers who responded indicateda background in education; the rest of the respondents became trainers after being managers,technical specialists, or salespeople. Such diverse backgrounds inevitably lead to variations inimportant competencies such as program planning, instructional methods, delivery techniques,training evaluation, and appropriate responses to the needs of adult learners (Henschke, 1991;McLagan, 1983, 1989).

Adult educators, for example, strongly favor specific methods that are consideredappropriate for use in adult learning situations. Methods that involve learner participation in theplanning of educational activities, respect for learners' backgrounds and needs, transformationof instructors from "know all" experts to facilitators of learning, and the instructor's true desireto help learners identify and self-direct their learning activities are all important considerationswhen working with adults in learning situations (Knowles, 1980). The practical applications ofthe emerging literature in the field of adult education suggest there are a variety of instructionalmethod alternatives for trainers who deliver corporate training programs Moreover, the fieldof educational psychology, using social learning/self-efficacy theories, gives further insight intothe motivation and decision-making process of trainers. To those charged with the responsibilityof providing professional development opportunities for corporate trainers, it would be usefulto be able to anticipate and respond to the criteria trainers use when deciding to adopt or rejecta proposed instructional method. To date, the fields of adult education, educational psychology,and corporate training have not empirically identified the criteria that trainers use to make thesedecisions.

Given the advances in technology as well as advances in our understanding of bothadult learning theory and social learning/self-efficacy theory, professional development forcorporate trainers is becoming increasingly critical. Yet, despite the enormous financial andhuman investments made in the field of corporate training, the training profession cannot

© copyright, C. Hickman, 1995

18-2

13

identify, let alone predict or use to its advantage, those criteria that trainers themselves use inselecting instructional methods. The literature that does exist relative to "training trainers" oftenprescribes how training should be conducted, as opposed to analyzing the decision-makingprocesses that trainers actually use in selecting methods. This gap in knowledge needs to beaddressed so that professional development for trainers can occur in a fashion that enhances thelikelihood that trainers will respond favorably to the development efforts used.

Theoretical Framework

Social Learning/Self-Efficacy Theory. Bandura's social learning theory, first proposed in1977, explains human behavior in terms of cognitive, behavioral, and environmentalcomponents. It also explains human behavior in terms of reciprocal determinism, whichstipulates a "reciprocal interaction between cognitive, behavioral, and environmentaldeterminants" (Bandura, 1977a, p. 204). Unlike previous views that considered behavior afunction of the person and the environment, reciprocal determinism emphasizes the mutuallyreciprocal roles of behavior, the person, and the environment on subsequent behavior. Peopleare therefore neither "victims" of the environment, nor are they totally free agents to becomewhatever they choose (Bandura, 1986).

Using the social learning theory model, it was theorized that corporate trainers aremarkedly influenced in their decision to accept or reject a proposed instructional method by 1)their conceptions and beliefs regarding the innovation, including their personal preferencestoward instructional methodologies; 2) the behavior required and self-perceptions of their abilityto perform the innovation successfully; and 3) the anticipated environmental outcomes ofperforming the innovation. Examples of environmental outcomes that would obviouslyinfluence both cognitive and behavioral activities include anticipated support (or non-support)from peers and supervisors, effective instruction and/or modeling (or lack thereof) of the properuse of a proposed method, and expected learning outcomes.

Self-efficacy theory, a subpostulate of social learning theory, is also applicable here.Within the context of Bandura's social learning theory, behavior change is hypothesized to occurthrough cognitive processes that are mediated by experiences of mastery stemming fromsuccessful performance. Bandura distinguishes between "efficacy expectations" and "outcomeexpectations" as two distinct and primary (although not exclusive) determinants of behavior(Bandura, 1977a, 1977b, 1986). An "efficacy expectation" is defined as "the conviction that onecan successfully execute the behavior required to produce the outcomes" (Bandura, 1977a, p.79). An "outcome expectation" is defined as "a person's estimate that a given behavior will leadto certain outcomes" (Bandura, 1977a, p. 79).

It is Bandura's contention that levels of self-efficacy determine whether behavior changeor coping strategies will be attempted or initiated, the amount of effort that will be expended,and the amount of time the effort will be sustained in the face of failures or other obstacles(Bandura, 1977b). Regarding the relevance of social learning/self-efficacy theory to educationalthinking, Ashton (1985) found that a "teachers' sense of efficacy [is] a critical construct inexplaining teaching motivation. It is expected to influence the teachers' choice of instructionalactivities, the amount of effort they expend in teaching, and the degree of persistence theymaintain when confronted with difficulties" (p. 144).

Applications of self-efficacy and social learning theory are fairly new in the trainingprofession. Consequently, only a few scientific studies have emerged that thoroughly explorethe role of self-efficacy in explaining behavior change variance relative to a training program.Indeed, there is no empirical evidence that explains behavior change, from a self-efficacy

18-2

14

perspective, in the corporate training population (Gist, 1986). However, social learning/self-efficacy theory has been successfully applied in studies identifying the criteria used in selectinginstructional methods in similar populations: elementary and secondary teachers (House, 1992);adult educators (Henscbke & House, 1989); and community college faculty (Lindstrom , 1990).

Research Questions and Hypothesis

This research was conducted in two stages and constituted two separate studies. In the firststage, the actual criteria corporate trainers employ in the selection of instructional methods hadto be identified. Therefore, the three research questions addressed:

1. Research Question 1. What criteria do trainers apply in the selection of instructionalmethods?2. Research Question 2. With what relative frequencies do trainers evaluate the criteria?3. Research Question 3. What demographic characteristics of trainers, if any, predictmethod selection criteria?

Given the answers to the first three research questions, an additional sample of thecorporate training population was obtained to further study instructional method adoptioncriteria from a social learning / self - efficacy theory perspective. The criteria were divided into twofactors, Intrinsic and Extrinsic Issues. The task was to determine if the corporate trainers usedin the two samples applied common intrinsic and extrinsic criteria with equal frequency, andwhether or not there were conceptual differences between corporate trainers and other teachingpopulations regarding intrinsic and extrinsic issues.

For the second stage of the study, the following hypothesis was tested: The criteria usedby corporate trainers to evaluate potential methods of instruction are attributable to two factorsas follows: Factor 1: Intrinsic Issues (Qualitative Trainee Outcomes, Trainer Enthusiasm, andTrainer Self-Efficacy) and Factor 2: Extrinsic Issues (Approval, Concept Versatility,Practicality of Implementation, and Quantitative Trainee Outcomes).

Methodology

The planning for the first part of the study occurred in three stages from October 1988 throughMarch 1989. During the first stage, the Method Acceptance Scale for Teachers (House, 1988),which was used to identify instructional method adoption criteria in elementary and secondaryteachers, was reviewed; Wording was changed to reflect the particulars of the corporate trainingpopulation. Additionally, items were added after consulting experts and the literature as topossible variables specific to the corporate training population. During the second stage, theinstrument was piloted to 10 experts who were asked to complete the questionnaire and offersuggestions for clarity, additions, and deletions. During the third stage, the instrument wasadministered to 372 trainers. These 372 trainers represented 33 companies in 30 cities in 18states. Two hundred ninety-seven (84%) respondents were full-time trainers, whereas 57respondents (15%) were part time. Almost half of the respondents (179: 49%) had 1-5 yearsexperience, but the mean number of years experience for the whole sample was 7.61 years (S.D.6.93 years). The respondents were fairly evenly divided by gender: 217 (58.3%) males and 153(41.1%) females.

18-2 15

The research instrument, renamed the Trainer Method Acceptance Scale (House &Hickman, 1989) had two parts. In Part One, respondents were asked how frequently theyconsider each item as they evaluate new approaches to instruction. Response options were"Never," "Rarely," "Sometimes," and "Often." Part Two asked for biographical data and wastitled "About You." Respondents indicated whether or not they currently instruct trainees, andthe level of their trainees in the organization; predominant content area taught; whether theywere full or part time; their age, sex, and years of experience as a trainer; number of previoustraining positions held; highest degree earned; whether or not they trained in-house; how theyperceived their training skills; previous training as a trainer; and the amount of enjoyment theygot out of training.

The data were subjected to an exploratory, principal component factor analysisfollowed by varimax rotation. The initial analysis included all 72 items and required thatextracted factors possess eigenvalues greater than one in magnitude. Thirty-six principalcomponents met this criteria. A scree plot indicated, however, that between six and eight factorsprobably could be clearly differentiated. Rotated factor loadings revealed some ambiguity bothfor item factor assignments and latent trait meanings. Items were gradually eliminated toimprove the clarity of item assignments and latent traits. The final factor solution included 42of the original 72 items (58.3%) and identified six distinct factors (see Table 1).

In the second phase of the research, these six factors were analyzed with an additionalsample of corporate trainers during the fall of 1992 (n=228). This time, the six distinctconstructs (factors) identified in the original research were grouped into two factors, labelledIntrinsic and Extrinsic Issues in accordance with Bandura's (1986) social learning/self-efficacytheory. Moreover, the TMAS instrument was amended to incorporate specific measures of self-efficacy, and a seventh construct (named Trainer Self-Efficacy) was hypothesized to be anIntrinsic Issue. These data were analyzed by maximum likelihood confirmatory factor analysis,using the LISREL VI program.

The statistical analysis consisted of a goodness-of-fit comparison between ahypothetical model and a null model. To reject the null counterpart of the research hypothesis,two sets of standards were applied. The first set of standards was statistical. Significant (alpha= .05) positive lambda loadings were required and a significant incremental Chi-square valuewas required between the full and null model. The second set of standards was practical. Thesingle practical index was a delta coefficient of greater than or equal to .90. This value wouldindicate that 90% of the total item variance was accounted for by the full model.

Results

During the first stage of the research, six conceptually distinct factors that corporate trainers usewhen deciding to select an instruction method were identified. Table 1 identifies these factors,their respective eigenvalues, and percents of variance.

Table 1

Factor Analysis of Training Method Acceptance Scale: Identified Factors with Eigenvalues >1

Factor Eigenvalue Percent Variance

Qualitative Trainee Outcomes(Will it increase trainee learning?) 8.18 18.6

18-2 1 :6

Table 1 Continued

Factor Eigenvalue Percent Variance

Trainer Enthusiasm(Trainer motivation and willingnessto try method) 4.14 9.4

Approval(Administration and/or peer sanction) 2.57 5.8

Concept Versatility(Can I use the method with manydifferent curricular areas?) 1.87 4.2

Practicality of Implementation(Do I have necessary materials andguidance to try this new method?) 1.73 3.9

Quantitative Trainee Outcomes(Will there be measurable results interms of trainee learning?) 1.52 3.4

On a scale of 1-4 (1=Never; 2= Rarely, 3=Sometimes, 4=Often), raw score means werecalculated by factor and are identified in Table 2:

Table 2

Raw Score Means by Factor

Factor Mean

Qualitative Trainee Outcomes 3.7Practicality of Implementation 3.3Trainer Enthusiasm 3.0Concept Versatility 3.0Quantitative Trainee Outcomes 2.8Approval 2.7

No statistical tests are reported because the large sample size virtually ensures that eachsuccessive mean difference is significant. Although the sampling process for this research doesnot support firm generalizations, it is reasonable to hypothesize that, for many trainers, methodsof instruction are most often selected because of trainer predictions of the effects these methodswill have upon trainee affect and cognition. The second most frequent reason for selection ishow practical the new method is perceived by the trainer to be, and the third most frequentreason for selection is a trainer's prediction of his or her own affective response in using the

method.

18-2

In response to the third research question ("What demographic characteristics oftrainers, if any, predict method selection criteria?"), stepwise multiple regressions of the sixfactors were performed on the 20 variables constituting trainer demographics. Table 3 reportsthose variables that were statistically related to the factors. While there appear to be a fewstatistically significant variables per factor, the practical significance is dubious in light of thelow R-squares and correspondingly low percentage of variance explained.

Table 3

Multiple Regressions of Factors on Demographic Variables (only significant variable[sl cited)

FACTOR VARIABLE(S) t p Adj. It2

Qualitative Trainee Educ. Level .062 .234 4.2 <.000 .05Outcomes

Trainer Enthusiasm Gender -.201 -.208 -3.7 <.000Instruct trainees? .290 .161 2.9 <.000 .06

Approval Years Experience -.014 -.167 -3.02 <.005Employed by Same Co.as Trainees .327 .156 2.8 <.05Inst. 1st Level Sup. -.185 -.141 -2.6 <.05 .07

Concept Gender -.150 -.149 -2.7 <.05 .04Versatility

Practicality of # Previous Positions .039 .128 2.26 <.05 .01Implementation

Quantitative Trainee Content Area -.077 -.198 -3.56 <.000Outcomes Trainees are Members

of Other Company .310 .151 2.695 <.05Full Time vs. PartTime .264 .133 2.392 <.05 .06

Consequently, the multiple regressions suggest that the demographics tested are notimportant predictors; i.e., the population is homogeneous relative to the six factors. Since itwould be unreasonable to assume that 372 trainers from 33 companies in 30 cities among 18states all had similar education, background, and philosophies, these data suggest that trainersadopt similar criteria for adopting training methods after entering the profession..

The third research question pertained to the extent to which the factors could be reliablymeasured. Reliabilities (internal consistencies) for the six TMAS factors were estimated throughCronbach's alpha coefficients. Respectively, the values are: Qualitative Trainee Outcomes = .84;Trainer Enthusiasm = .80; Approval = .81; Concept Versatility = .69; Practicality ofImplementation = .73; and Quantitative Trainee Outcomes = .64. Consistent with statisticalexpectations, there are direct relationships between the order in which factors were extracted,numbers of items constituting each factor, and the magnitudes of corresponding coefficients.

18-2

is

Coefficient magnitudes generally indicate, however, moderate to strong relationships betweenitem and raw scores.

The second part of the study is generally considered to be more academic, than practicalin nature. Hence, only the salient features will be discussed.

In accordance with Bandura's social learning/self-efficacy theory, it was hypothesizedthat the original six factors (plus one called Trainer Self-Efficacy) were actually constructs thatconstituted two factorsIntrinsic and Extrinsic Issues. This model was tested with a sample of228 trainers representing 107 companies in 81 cities in 35 continental U.S. states. The responseto variable ratio was 32.57:1, well above the 30:1 minimum desired (Hair, Anderson, &Tatham, 1987).

The TMAS instrument was also revised to delete unreliable items and incoherentfactors. The final TMAS instrument consists of 44 items designed to assess the sevenconstructs. The items assessing each construct are arranged randomly throughout the TMASinstrument.

Construct Means, Standard Deviations, and Reliabilities. Construct scores for eachsubject were computed as the sums of corresponding item scores divided by the respectivenumber of items. Therefore, the construct scores were expressed in the same units as itemscores, i.e., using a scale of Never -1, Rarely=2, Sometimes=3, and Often=4. Table 4 outlinesthe construct means, standard deviations, and alpha reliabilities for the seven constructsidentified in the TMAS.

Table 4Construct Means, Standard Deviations, and Reliabilities

Construct Factor Mean StandardDeviation

Alpha(Reliability)

Qualitative Trainee Outcomes Intrinsic 3.8 0.30 .8328Trainer Enthusiasm Intrinsic 3.0 0.56 .8525Trainer Self-Efficacy Intrinsic 3.5 0.57 .7656Approval Extrinsic 2.8 0.58 .7838Concept Versatility Extrinsic 3.0 0.47 .6708Practicality of Implementation Extrinsic 3.4 0.43 .6249Quantitative Trainee Outcomes Extrinsic 2.8 0.72 .5903

The rather academic exercise of simplifying the model to two distinct factors (Intrinsicand Extrinsic Issues) comprised of the seven constructs listed above was not empiricallyconfirmed using LISREL. This hypothetical model was tested against a null model with alllambda parameters set at zero. Two conditions needed to be met to confirm the hypotheticalmodel. First, positive lambda t-values were required to be statistically significant at alpha =.05.Second, to achieve practical significance, the hypothetical model needed to achieve a goodness-of-fit delta value of z .90 in comparison to the null model.

All t-values were positive, indicating positive lambda values, and exceeded the .05statistical criterion. The incremental chi-square values exceeded the .05 statistical criterion.However, the goodness-of-fit delta value failed to achieve the a priori practical assessmentcriterion of .90; therefore the hypothesized model was not confirmed.

18-2 19

Discussion

This study proposed to identify the constructs that corporate trainers consider important whendeciding whether or not to actually try a new instructional method in their classrooms. Sevenconstructs have been found to be statistically significant to trainers when makingmethodological decisions. These results can have profound implications for those who arecharged with the professional development of trainers. By suggesting instructional changes onthe basis of a congruence between the beneficial aspects of the method with the trainer's ownvalue system, chances of successful implementation increase.

In descending order of construct mean magnitude, the following constructs appear tohave influence over trainers in their decision to adopt or reject an instructional innovation:

1.) Qualitative Trainee Outcomes. Questions almost always (Mean = 3.8) asked bytrainers: Will the trainees enjoy it? Will it improve my trainees' productivity? Will it enhancetransfer back to the job? Will it increase trainee interest in the topic? Will it improve theirenthusiasm? Will it aid in trainee comprehension? Will it actively involve the trainees? Will itchallenge them?

2.) Trainer Self-Efficacy. Questions that are "sometimes" to "almost always"(Mean=3.5) asked by trainers: Can I use the method skillfully? Can I use the methodcompetently? Can I perform the technique?

3.) Practicality ofimplementation. Questions that are "sometimes" to "almost always"(Mean=3.4) asked by trainers: Has it been successfully implemented elsewhere? Is competenthelp available if I need it? Do I have, or can I obtain, the necessary materials? Was it presentedto me skillfully? Do I have sufficient information to try it?

4.) Concept Versatility. Questions "sometimes" (Mean=3.0) asked by trainers: Will itincrease my opportunity to work with other trainers? Is it appropriate, or can I adapt it, to thelearning styles of my trainees? Is one of my colleagues familiar with it also? Can it be used withvarious levels of personnel within the organization? Is it compatible with other training methodsused? Can I use it in different content areas?

5.) Trainer Enthusiasm. Questions "sometimes" (Mean=3.0) asked by trainers: WillI enjoy using it? Will it renew, maintain, or increase my interest in instructing? Will I find tryingit pleasant? Is it compatible with my teaching style? Will it satisfy my curiosity? Will it enhancemy motivation to train? Can I learn the method easily? Will trying it invigorate me?

6.) Approval. Questions "sometimes" (Mean=2.8) asked by trainers: Will mycolleagues become interested in it? Will my supervisor approve? Will it benefit my career? Willmy peers approve? Will senior management approve? Will it positively affect my performanceevaluation?

7.) Quantitative Trainee Outcomes. Questions "sometimes" (Mean=2.8) asked bytrainers: Will it improve employees' performance appraisal ratings? Will it improve traineecompetence on corporate promotion-related criteria? Will it enhance trainee corporatecitizenship?

Moreover, it is discovered that demographic variables of trainers, such as age, contentarea taught, number of years as a trainer, education level, full-time vs. part-time, nor numberof professional career-moves made, were not significant predictors of criteria used to try newinstructional methods. This is consistent with the literature which suggests that professionalstend to adopt group-like viewpoints, philosophies, and identification fairly quickly into the joband the demographic variables listed are rarely statistically or practically significant inpredicting professional attitudes.

Future research needs to further explore the role of a trainer's level of self-efficacy.Specially, we need to determine precisely how important, if at all, the construct of self-efficacyis in methodological decisions. We need to determine the extent to which trainers consciously

18-2

20

assess their level of self-efficacy regarding an instructional innovation. Do trainers readilydismiss suggestions and training regarding new methods if they suspect they cannot perform themethodand do they cognitively attribute their dismissal to their inability to perform it? Or, dotrainers consciously assess their skill-level and remain open to self-efficacy building activitiesfor methods that may otherwise have merit?

Another important area of research would be to determine similarities and differencesbetween the criteria identified by American trainers and by trainers in other countries andcultures. This type of knowledge would be important when training trainers in other cultures,which is increasingly the case as Americans exchange technical information with Japanese,Russian, and Middle Eastern trainers.

Although this study failed to validate an ultimately-hypothesized two-factor structure,seven cognitive constructs were found to have statistical significance. Also, for the first time,self-efficacy was found to be an important construct with statistical significance, using itemsthat had face validity. These seven constructs can serve as a basis for further research to refineour knowledge of the cognitive antecedents of trainers' methodological decisions. These avenuesof research promise to increase our understanding of trainers and the criteria they employ whenmaking methodological decisions.

References

Ashton, P. (1985). Motivation and the teacher's sense of efficacy. In C. Ames & r. Ames (Eds.)Research on motivation in education, Vol. 2 (pp. 141-171). Orlando: Academic Press,Inc.

Bandura, A. (1977a). Social learning theory. Englewood Cliffs: Prentice-Hall, Inc.Bandura, A. (1977b). Self-efficacy theory: Toward a unifying theory of behavior change.

Psychological Review, 84(2), 191-215.Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory.

Englewood Cliffs: Prentice-Hall, Inc.Gist, M.E. (1986). The effects of self-efficacy training on training task performance. Academy

ofManagement Best Paper Proceedings, 46, 250-254.Hair, J.E., Anderson, R.E., & Tatham, R.L. (1987). Multivariate data analysis with readings

(2nd Ed.). New York: Macmillan Publishing Company.Henschke, J.A. & House, G.D. (1989). Report of the study: Development ofpolicy for hiring

and maintaining ALSP instructors, City of Chicago. Unpublished manuscript.Henschke, J.A. (1991). Models for HRD practice: The academic guide. Alexandria: The

American Society for Training and Development.House, G.D. (1988). Factors underlying instructional method: Decision-making in

elementary teachers. Paper presented to the annual meeting of the AmericanEducational Research Association, New Orleans, Louisana.

House, G.D. (1992). Cognitive antecedents of instructional behavior in elementary teachers.

Unpublished manuscript.House, G.D. & Hickman, C.J. (1989). Training method acceptance scale for trainers: A

technical report. Unpublished technical report.Knowles, M.A. (1978). The adult learner: A neglected species. Houston: Gulf Publishing

Company.Knowles, M.A. (1980). The modern practice of adult education. Cambridge: The Adult

Education Company.

EST COPY MAMA

18-2

LE

Lindstrom, J. (1990). Instructional method evaluation criteria of community college teachers(Doctoral dissertation, University of Missouri, 1990). Dissertation AbstractsInternational, 51, 12A, 9109777.

McCollough, R.C. (1987). Professional development. In R.L. Craig (Ed.) Training andDevelopment Handbook. St. Louis: McGraw-Hill.

McLagan, P.S. (1983. Models for excellence: The conclusions and recommendations of theAS77) training and development competency study. Alexandria: The American Societyfor Training and Development.

McLagan, P.S. (1989). The models. Alexandria: The American Society for Training andDevelopment.

Exploring The Use Of Training Games

James J. Kirk & Hal ShoemakerWestern Carolina University

This investigation explored differences in the use of games bytrainers in 82 companies. The results indicate that games areused for a variety of training and group process purposes.Trainers who spent a larger percentage of their training timeon gaming activities tended to use games more often 'togenerate ideas/solutions' and trainers who learned tofacilitate games solely through experience tended to usegames more often to 'introduce new concepts.'

Many educators and trainers consider games an effective instructional strategy. By "games" trainers aretypically referring to activities requiring participants to perform tasks, play assigned roles, follow groundrules, and/or strive to reach a designated goal. Learners frequently find games a desirable departure fromtraditional training lectures. The challenge games present often gets the heart racing, blood rushing,andadrenaline flowing. The result is a "natural high." Since the outcomes of games are usually in doubt, theyhave a tendency to captivate trainees' attention. A psychological need for closure makes people want toknow how things are going to turn out (i.e., who will become the winners). Furthermore, because mostgames are played in the company of other individuals they also help satisfy the human need for socialinteraction.

To adequately prepare students for HRD practice it is important for professors to provide them anunderstanding of how training games are currently being used in the workplace. Therefore, the aim of thisstudy sought answers to the following research questions: (1)Is there a significant difference in theinstructional purposes for which training games are used by trainers? (2)Is there a significant difference inthe group process purposes for which training games are used by trainers? (3)Is there a significant differencein the instructional purposes for which training games are used by trainers with respect to a trainer's age,gender, years of training experience, training specialty, source of training games, supervisor's views on theeffectiveness of games, the percentage of training time a trainer spends on gaming activities, and how atrainer learned to facilitate games? and (4)Is there a significant difference in the group process purposes forwhich training games are used by trainers with respect to a trainer's age, gender, years of trainingexperience, training specialty, source of training games, supervisor's views on the effectiveness of games,the percentage of training time a trainer spends on gaming activities, and howa trainer learned to facilitategames?

Related Literature

An early use of training games was to train military personnel (Taney & Unwin, 1969). Chess wasamong the first games used for this purpose and led to other tactical games involving map maneuvers(Thomas, 1957). Near the end of the nineteenth century some military training games involved an elaborateset of rules and used charts to depict every conceivable roll of the dice. Other games involved more real-lifesituations using military personnel to play out various scenarios. The use of such gaming simulationsgrew to be a standard part of military training throughout the world (Taney & Unwin, 1969).

During the early part of the 20th century, the growth of industry created a need for more training.Training typically came in two forms--on-the-job training and classroom instruction. Over time on-the-jobtraining proved to be slow in delivery and restrictive in content. It was often difficult for workers to transferback to the job site information presented in classroom lectures. Taking their cues from military trainers,trainers in business and industry began to see games as a means of dealing with the limitations of both on-the-job training and classroom instruction. In 1956 the American Management Association developed the"Top Management Decision Simulation" which became the first widely used business game in the UnitedStates (Taney & Unwin, 1969).

copyright, Kirk & Shoemaker, 1995

18-3

23

According to Thatcher (1990) organizations currently use games fora variety of training purposes(e.g., to teach envisioning, negotiations, and decision-making skills). Games are also used to developentrepreneurial expertise (Prohask & Frank, 1990) and to help participants gain a sense of camaraderie(Gamson, 1994). Trainers have found that games can be an exciting and informative activity leading toboth high cognitive and high emotional involvement in the subject matter at hand (Thatcher, 1990;MacCrimmon & Wehrung, 1984). Participants actually begin to act and feel like characters in a real lifesituation (Thatcher, 1990). They subsequently begin to make decisions very similarly to decisions theymight actually make in real life (Prohaska & Frank, 1990).

While there is little doubt that games have gained acceptance as a viable training tool, theireffectiveness as an instructional strategy appears less certain. Some researchers hold that games are neithermore or less effective than other training methods (Boocock, 1994; Cherryholmes, 1966; Dukes, 1994;Pierfy, 1977). However, there is some evidence that games heighten participants' interest more thantraditional instructional methods (Cherryholmes, 1966; & Pierfy, 1977). This heightened interest issometimes attributed to the more active involvement of learners while participating in gaming activities(Petranek, Corey, & Black, 1992; & Coleman, 1980). Other research has found that games contribute tohigher affective and behavioral learning compared to traditional instructional modes (Bredemeier &Greenblat, 1981) and better enable participants to fit information into a real life context (Coleman, 1980).The studies of Petranek, Corey, & Black (1992) and Foster, Lachman, & Mason (1988) have found gamesto be a more effective teaching tool in the understanding of human behavior.

Method

Two hundred fifty questionnaires along with cover letters were mailed to randomly selected trainers listed inASTD' s Whose Who In Training & Development. Eight-two usable surveys were returned giving thestudy a return rate of 33%. The questionnaire contained 23 questions (i.e., 13 multiple-choice, 4 rank-ordering, and 6 fill-in-the-blank type items). Questions inquired into trainers' ages, gender, trainingexperience, training specialty, use of training games, and immediate supervisor's views on the effectivenessof games as a training strategy.

Prior to testing for statistical significance, descriptive statistics were generated for the variablesinvestigated. Interval data such as age, years of training experience, and percentage of training time spenton gaming activities were converted to categorical data A series of multiple group Chi-Square tests weresubsequently performed to test for significant differences between expected and observed frequencies. P s.05 was used as the level for determining statistical significance.

Results

An examination of the descriptive statistics revealed that respondents were primarily female (68%) with amean age of 38 years. On average they had spent 9 years in the training field. Twenty-eight percentreported a training specialization in management/supervisory training, 15% in technical training, 15% inhuman relations training, and 6% in career development. Seventy-six percent of the trainers claimed theirimmediate supervisors considered training games just as effective or more effective than otherinstructional/group process strategies.

Fifty -four percent of the respondents said that they learned to facilitate games solely throughexperience (i.e., without any coaching on the part of colleagues or the benefit of any classroom instruction).They reported spending 18% of their training time facilitating gaming activities. Fifty-eight percent of thegames they used on a regular basis were created by the trainers themselves. Forty-two percent of thetrainers said that they spend at least 15 minutes debriefing players at the end of each game.

The Chi-Square test employed to answer question one (i.e., Is there a significant difference in theinstructional purposes for which training games are used by trainers?) revealed no significant differences inthe ranking of five instructional purposes (reinforce previously covered information, teach new concepts,introduce new ideas, measure what participants have learned during training, and assess what trainees alreadyknow about a subject prior to training). Mean rankings for the five instructional purposes appear in TableOne.

EST COPY AVAIIA

18-3 24

LE

Table One: Mean Rankings For Instructional PurposesN=82

Mean Ranking Purpose2.354 Reinforce previously covered information.2378 Teach new concepts.2.415 Introduce new ideas3.622 Measure what participants have learned during training.3.804 Assess what trainees already know about a subject prior to training.

*Items ranked 1-5 with 1= "your most frequent instructional purpose for using a game and 5 = "your leastfrequent instructional purpose for using a game."

The Chi-Square test employed to answer question two (i.e., Is there a significant difference in thegroup process purposes for which training games are used by trainers?) revealed no significant differences inthe ranking of five group process purposes (generate ideas, break the ice, motivate participants, buildrapport among group members, build group cohesion). Mean rankings for the five group process purposesare presented in Table Two.

Table Two: Mean Rankings For Group Process PurposesN:t2

Mean Ranking Purpose2.39 Generate ideas.2.866 Break the ice.2.89 Motivate participants.3.024 Build rapport among group members.3.451 Build group cohesion.

*Items ranked 1-5 with 1 = "your most frequent group process purpose for using a game" and 5= "yourleast frequent group process purpose for using a game."

The Chi-Square test conducted to answer question three (i.e., Is there a significant difference in theinstructional purposes for which training games are used by trainers with respect to age, gender, years oftraining experience, a trainer's specialty, a trainer's source of training games, the percent of training time atrainer spends on gaming activities, how a trainer learned to facilitate games, and a trainer's supervisor sviews on the effectiveness of games?) revealed one significant difference. Trainers who learned to facilitategames solely through experience used games to introduce new concepts significantly more often than didtrainers who learned to facilitate games through some on-the-job coaching or by attending classes. SeeTable Three below for ranking frequencies for using games to introduce new concepts (Chi Square =10.478;P = .0331).

Ranking

Table Three: Ranking Frequencies.For Using GamesTo Introduce New Concepts

Learned To Facilitate Games Learned To Facilitate Games ViaSolely Though Experience On-The-Job Coaching &

Attending ClassesRanked #1 18 7Ranked #2 8 13Ranked #3 10 9Ranked #4 2 7Ranked #5 6 2

The Chi-Square test employed to answer question four (i.e., Is there a significant difference in thegroup process purposes for which training games are used by trainers with respect to age, gender, years oftraining experience, a trainer's specialty, a trainer's source of training games, the percent of training time atrainer spends on gaming activities, how a trainer learned to facilitate games, and a trainer's supervisor'sviews on the effectiveness of games?) also revealed one significant difference. Trainers who spent 25% ormore of their training time facilitating gaming activities used games to generate idea/solutions significantlymore often than trainers who spent 12% or less of their training time facilitating gaming activities. See

18-3

25EST COPY AVAIIABLE

Table Four below for ranking frequencies for using games to generate ideas/solutions (Chi Square= 12.526;P = .014).

Table Four: Ranking Frequencies For Using GamesTo Generate Ideas/Solutions

N=82Ranking Trainers Spending

1-12% of TimeTrainers Spending

25% or More of Time

Ranked #1 8 14Ranked #2 6 5Ranked #3 9 3Ranked #4 6 2Ranked #5 8 0

Conclusions:

The major significance of the study is that it supports the notion that training games have become a veryimportant part of training. Respondents reported that on average they spend 18% of their training timefacilitating gaming activities. A second significance of the study is that it highlights the variety ofpurposes for which trainers currently use games. The lack of any significant differences found among thefive instructional purposes for which training games were used suggests that the use of games may haveexpanded beyond some of the traditional uses such as to "teach new concepts" and to "reinforce previouslycovered material." Today trainers are just as likely to use a game to "pre- and post-test a learner's knowledgeof subject matter content." The use of games for group process purposes also appears to have gone beyondsuch traditional uses as "breaking the ice" and "motivating trainees." Games are equally likely to be usedfor "generating ideas" and "building rapport among group members."

Two conclusions can be drawn from the statistical differences found in the study. First, the findingthat trainers who spend a larger percentage of time on gaming activities tend to use games more often "togenerate ideas/solutions" probably has something to do with the nature of the games they are facilitating.Games that are used in problem-posing and problem-solving may simply last longer than games used tobreak the ice or to motivate participants. A more frequent use of such time consuming games may drive upthe overall amount of training time a trainer spends on gaming activities. The finding that trainers wholearned to facilitate games solely through experience tend to use games significantly more often to"introduce new concepts" suggests that trainers who learned to facilitate games through less traditionalmeans may be more apt to use games for less traditional purposes. It has been more customary for gamesto be used to "teach" or "reinforce" new concepts as opposed to "introducing" new concepts (Matcher,1990).

Implications:

The findings of the research have implications for HRD academes and trainers, as well as the creators oftraining games. The large portion of training time being taken up by gaming activities (approximately18%) and the large percentage of trainers receiving no structured training of any kind on game facilitation(80%) suggests that there may be a need to offer more college courses on the theory and use of instructionalgames. In addition, the large amounts of time being spent on training games to achieve a wide range ofpurposes strongly suggest a need to more thoroughly research their effectiveness in achieving suchpurposes. As for trainers themselves, if they wish to use games for group problem solving (e.g., gamesintended to generate ideas and/or solutions) they may need to be prepared to devote alarger percentage oftheir overall training time for gaming activities. These types of games may require more time to facilitate.Finally, the current acceptance and use of games as a viable training tool is likely to afford creative writingopportunities for academes in a variety of disciplines. In addition to the games themselves, professors andtrainers are likely to be in need of books that discuss both the theory and use of games. To date, thisinstructional strategy has received little attention by human resource developers.

18-3 26

References

American Society for Training and Development. (1993). ,1993 Who's who in training and development.Alexandria, VA: American Society for Training and Development.

Boocock, S.S. (1994, June). Johns Hopkins Games Program. Simulation & Gaming. 172-178.Bredemeier, M., & Greenblat, C. (1981). The educational effectiveness of simulation games. Simulation &

Games, .12, 307-331.Cherryholmes, C. (1966). Some current research on effectiveness of educational simulations: Implications

for alternative strategies. American Behavioral Scientist, 10, 4-7.Coleman, J.S. (1980). Academic games and learning. In R. Stadskley (ed.), Handbook of Simulation

Gaming in Social Education Part I: Textbook (15-25). University, AL Institute of HigherEducation Research & Services.

Dukes, R.L. (1994, June). An appraisal of simulation and gaming on the 25th anniversary of S&G.Simulation & Gaming, 193-197.

Engin, A.W., & Klein, I.R. (1975). The effectiveness of a simulation technique as an integral part of aschool psychology training program. Journal of School Psychology, 12, 171-184.

Faria, A.J. (1989). Business gaming: current usage levels. Special Issue: Management DevelopmentReview, Journal of Management Development, 8 58-65. -

Foster, J.L., Lachman, A.C., & Mason, R.M. (1980). Verstehen, cognition, and the impact of politicalsimulations. Simulation & Games, a 223 -241.\

Gamson, W.B. (1994, June). SIMSOC at 30. Silver Anniversary Issue. Simulation & Gaming. Z. 207-211.

Hemphill, J.K., Griffiths, DE, & Frederiksen, N. (1962). Administrative Performance and Personality.New York: Bureau of Publications, Teachers' College, Columbia University.

Jaffe, E., & Nebenzahl, I.D. (1990, June). Group interaction and business game performance. Simulation &Gaming 21 133-145.

MacCrimmon, K., & Wehrung, D. (1984). The risk in-basket. Journal of Business, 2, 367 -387.Petranek, C.F., Corey, S., & Black, R. (1992, June). Three levels of learning in simulations:

Participating, debriefing, and journal writing. Simulation & Gaming a. 174-185.Pierfy, D. (1977). Comparative simulation game research: Stumbling blocks & stepping stones.

Simulation & Games a, 225-268.Prohaska, C.R., & Frank, E.J. (1990, March). Using simulations to investigate management decision

making. Simulation & Gaming, 21, 48-57.Tansey, P.J., & Unwin, D. (1969). Simulation and Gamine Education. Toronto: MethuenEducational Ltd.Thatcher, D. C. (1990, September). Promoting learning through games and simulations. Simulation &

Gamine, 262-273.Thomas, C.J. (1957). The genesis and practice of operational gaming. Proceedings of the First

International Conference on Operational Research, (p, 66). Baltimore: Operations ResearchSociety of America

2718-3

Structured On-the-Job Training: Domain and Factors

Jong Cheul YangPOSCO Management Research Institute, Korea

In order to identffi, on-the-job training, this paper critiques previous definitions ofstructured on-the-job training (SOX) and defines it based on the critiques. Afterdiscussing target clients of on-the-job training (OJT) and the realtionship between OJTand other training methods, a SOJT process model is provided Fianlly, factors whichshould be considered before desinging a SOJT program are discussed

There is an ancient proverb: Knowledge is a treasure but practice is the key to it. Thisemphasizes the importance of transferring knowledge into use. Off has been recognized for itspotential for high transferability compared with other training methods because of an identicalor similar setting between work and training. Malcolm (1992), stating the results of aHoneywell study, found that "About 80 percent of all critical job skills [of managers] areacquired...on the job" (p. 58). OJT has been recognized as the most common form of trainingin businesses and industries (Lewick-Wallace & Jack, 1988). Carnevale and Gainer (1989)viewed on-the-job training as "the principal means by which technical, economic, strategic, andregulatory changes are gradually integrated into the workplace" (p. 15).

In spite of its pervasive use and criticality, there are serious concerns about Off.Learning by doing is hit-or-miss training with no guarantee of what tasks a worker will learn(Connor, 1983). Off has been regarded as cheap and easy training to implement without anyplanning (Goldstein, 1993). This kind of training may waste work resources. DeJong (1994),Jacobs (1992), and Rothwell (1990a & 1990b) emphasized the importance of structuring Off,thus, differentiating it from unstructured OJT which is "haphazard, incomplete, and toodisruptive in the workplace..." (Jacobs, 1992, p. 500).

Because of these problems, it is necessary to identify the domain of OJT before tryingto structure it. The domain or the subject area of Orr can be identified through defining it,fmding what steps it comprises, and specifying the relationship with off-job-training (OFFJT).

Definitions of SOJT

The importance of structuring or organizing Off has been emphasized from several decades ago(Engel, 1958; Goldstein, 1993; Nadler, 1979). Engel (1958), cited by Kruger (1985, p. 11),defined OJT as "systematic learning through the actual performance of an occupation in theenvironment in which the job will be performed....a process of accomplishing the steps in thewritten training program, over a period of time, together with an evaluation of accomplishmentsand progress at noted intervals."

Jacobs has made an effort to identify SOJT's boundary. His definitions of SOJTchanged several times during the last few years:

"the one-to-one process of providing the knowledge and skills to perform aspecific task within a job" (1992, p. 500);

"the planned process of transferring expertise from experts to novices in thework setting" (1992, p. 2);

"the planned process of developing task-level expertise by having anexperienced employee train a novice employee at or near the actual worksetting" (Jacobs & Jones, 1995).

cbopyright, Jong Cheul Yang, 1995

'One-to-one process' in the first definition implies that the trainee numbers should belimited to less four in most cases (Jacobs, 1992) because OJT includes close observation andguided practice. But Mikes & Sullivan's (1985) successful case study showed that theinteraction among trainees and other staff's involvement in the communication channel withtrainees is needed in an OJT program, depending on the task types and duration of the training.As Rothwell (1991) and Senge (1990) pointed to the importance of team learning, a wholework-unit may have to take an OJT program, especially when the tasks need interaction amongthe unit members.

In the second definition, the notion that expertise on a task is transferred from expertsto novice is not appropriate considering adult learning theories. Often expertise in adult learningis self-generated by learners themselves with facilitation from their instructor. Trainees are notnecessarily novices, especially when the training is designed to obtain context knowledge ratherthan to obtain subject matter expertise, for example, a newly hired school bus driver who mighthave been a commercial bus driver. What he or she needs is to become familiar withgeographical features on the bus route and the characteristics of his or her passengers, students.Adult learners should be analyzed to determine their expertise obtained through previousexperiences before attempts are made to train them.

SOJT does not have to be restricted task-level expertise which stated in the thirddefinition. In Mikes and Sullivan's (1985) study, the OJT content was the bank assistant-manager's job intertwined with many individual tasks. Except for simple and repeated tasks,it is more important to understand how to relate individual tasks to each other in a project orjob than to develop individual task expertise. Jacobs (1992) himself recognized that "someobservers view the 'project' as the most appropriate unit of work analysis, as opposed to thetask, which they believe does not represent the expectations of today's work-place" (p. 511).

Considering the above concerns, the present author defines SOJT as:Planned training to develop workers' level of procedural knowledge requiredin performing specific tasks/projects primarily through observation and practiceon or near the job site guided by a expeienced line workers who hasinstructional competencies.In this definition, the key words are 'planned training,' observation and guided practice,

and procedural knowledge. Like other formal training methods, OJT should be plannedconsidering work schedule, human resources, and so on. In the planning step, involvement oftop management as well as the line manager and the training coordinator will promote theeffectiveness and efficiency of the program (Sullivan & Miklas, 1985). Top managementinvolvement includes establishing the organizational policy which guarantees ongoing company-wide support (i.e., rewarding and/or relieving work load for Off practice).

There is a Korean maxim which emphasizes the effectiveness of observation in learning:Hearing one hundred times is not as good as observing one time. But a trainer should be awarethat his or her behaviors may be very automated and so fast that the trainee cannot follow it.The behaviors may not be able to be done without expertise obtained over a long time withrepeated performance. The OJT trainer should know the key portions of learning theories andlearner-centered instruction. In guided practice stage, OJT instructors should also supervise thetrainees performance for the safety of people and equipment.

The concept of procedural knowledge is based on Anderson's (1985) cognitive model.Anderson divided knowledge into two levels: (a) declarative knowledge about facts and things,and (b) procedural knowledge about how to do something. For example, declarative knowledgeabout my car includes the fact (there are the gear stick, the brake and the accelerator in thecontrol system) and description of how to change the speed. The procedural knowledge includesbeing able to change the speed actually. Anderson's (1987) interest was to fmd out that howprocedural knowledge is developed from declarative knowledge and how it is turned into fast,accurate and flexible routines of high performance.

Besides these cognitive abilities, psychomotor skill is also needed in performing tasks.Declarative knowledge and psychomotor skills are primarily developed through off-job trainingand/or self-directed training. Procedural knowledge connects the declarative knowledge to the

18-4

29

behaviors according to the context cues in the work setting, thus, OJT is needed to developprocedural knowledge. Figure 1 relates appropriate training methods to the several componentsof competencies.

Figure 1. Components of Competencies and Training Method

declarative knowledge (goal,principles, facts, and self-knowledge)

OJTOFFJT I procedural

knowledge

psychomotor skillsphysical skills

Self Learning

Target Clients of OJT Most authors who have studied OJT have dealt with simple tasksperformed by lower positioned production, lab or shop floor workers working with first-linesupervisors (Connor, 1983, 1985; Kruger, 1985; Jacobs, 1987; Wichman, 1989). Several bookson OJT have been written for line supervisors: On-the Job Training (Connor, 1983); One-to-One/Step-by-Step (Connor, 1988); and The Supervisor and On-the-Job Training (Broadwell,1995).

However, considering that the purpose of OJT is to develop procedural knowledge andskills, it is unnecessary to restrict the target population of OJT to lower positioned workers.According to Malcolm (1992), most critical job skills of managers are acquired on the job. Ifthat is true, there is a great need to develop SOJT programs for managers, even CEOs. Ifsuccessors to CEOs who are going to retire are appointed prior to their retirement, SOJT canbe used to help successors understand the whole company and its context cues to assist incarrying out management duty.

Relationship Between OJT and Other Training Methods Some authors (Mangum,1985) compared the pros and cons of training methods according to the situation, demand, andconstraints, primarily considering OJT with OFFJT dichotomously. This may be like comparingbread and butter, both of which have different functions and different dimensions andcharacteristics. Off and OFFJT should be integrated to compensate for their weaknesses andto strengthen them. Without obtaining declarative knowledge thorugh OFFJT, OJT cannot beeffective because trainees do not know what to do. Without OJT, the content knowledge cannotbe applied with high trafer rate on the job. Rackham (1979) reported a case of sales trainingwherein 35 sales persons' performance was not improved after conventional classroom teaching.The problem was solved with a program designed where managers in the branch were taughtmethods for analyzing selling skills and systemically training their sales people on the job.

SOJT Steps Earlier, Engel (1958), Gold (1981), Stokes (1966), reviewed by Kruger(1985), emphasized the importance of structuring OJT for an efficient process and effectiveresults. Kruger (1985) summarized the steps and specific actions of each step. The OJT stepsinclude: prior to instruction, instruction, presentation, application and follow-up. Jacobs (1995)provided a OJT process model: decide whether to use SOJT, analyze the tasks to be learned,select, train, and manage the trainers, prepare training modules, deliver the SOJT, and evaluateand troubleshoot the SOJT. These models include similar steps even though they used differentterminology.

The above models show the sequential procedure from selecting OJT as the trainingmethod to follow-up steps. Jacobs' Off process model is a single loop. But the procedure doesnot have to be sequential. When the same or a similar program had been implemented before,the analysis step can be skipped or little adjusted. According to the formative evaluation, theorder of steps can be switched or the same step can be repeated.

EST COPY AVAIL LE

18-4

30

The present author developed an instructional model consisting of six stages: analyze,design, develop, plan for implementation, deliver, and evaluate (Table 1). The personnelinvolved in each sub-step are included. Line experts are employees who have sufficientknowledge and skills to perform a specific job anywhere in the organization.

Some authors (Marquardt, 1976; Kruger, 1985) have pointed to a major problem inmany informal Off efforts: line experts, who are often charged with implementing OJT, arenot prepared in instructional knowledge and skills. One possible response to this problem is toencourage subject matter experts who are responsible for OJT to take a train-the-trainer programspecifically designed for OJT (Jacobs & Jones, 1995). Such a program would be strengthenedif competencies needed by line experts for structured on-the-job training (SOJT) were identified,along with those competencies needed by human resource development professionals to supportline experts who are doing SOJT. This model (Table 1) will be helpful to identify instructionalcompetencies needed for performing each step.

Table 1

SOJT Steps and Personnel Involved

Step Personnel

1. Analyze

2. Design

- specify organization goals MN*- determine training climate TP**- analyze the target tasks/task cluster and TP LE***required knowledge, skills, and attitude forthe performance.

- analyze trainee knowledge/skill levels, TP, LElearning characteristics, and personality

- present training needs and goals to management TP, LE,for approval MN

- set objectives- determine sub-steps of delivery (explain,show, part practice, and whole practice)

- determine delivery media (job guide, CBT,video, one-to-one, etc.)

- determine evaluation methods (written test,performance check list, on-the-jobperformance)

3. Development- develop learning materials- develop evaluation tools

TP,TP, LE

TP, LE

TP, LE

TP, (LE)TP, (LE)

- obtain feedback and revise TP, LE- pilot test and modify TP, LE

4. Plan for Implementation- check the operation schedule LE- plan training schedule (number of trainees, TP, LEnumber of trainers and staff personnel, tasks,condition, time, and equipment)

- set the criteria to select trainers, staffs, TP, LEand trainers

- select candidates for OJT instructor and TP, LEsupporting staffs personnel, and trainees

- provide train-the-trainer programs for line TP, LEexperts

18-4 31

(Table 1 continued)

Step Personnel

- arrange the workload of trainers supporting MNstaff, personnel, and trainees

- provide appropriate social and physical TP, LEsupport MN

6. Delivery- place tools and learning materials at TP, LEtraining site

- place trainees TP, LE- prepare learning place LE- deliver training according to design by LEexplaining, showing, guiding practice fromwhole to part to whole

- monitor the training process TP, MN- coach on dissatisfied parts of the trainees' .LEperformance

6. Evaluate- evaluate the trainees' performance as planned LE- evaluate the trainer's instructional TP,(MN)performance

- evaluate training materials TP, LE- evaluate training process TP, LE- document training record TP, LE- line manager/supervisors' evaluation on the LE, MNjob after training (one week, one month, six TPmonths...)

7. Follow-up- coach as needed LE- return to necessary earlier steps LE, TP- recognize trainees' completion of OJT MN, TP,

(managers' congratulation, certification, LEpromotion, or pay raise)

* MN: management, ** TP: training professional, *** LE: lineexperts

Factors in Structuring OJT

Many authors (Jacobs, 1995; Gold, 1981; Kruger, 1985; Rothwell, 1991b) asserted that Offshould be structured, pointing out the negative results of unstructured OJT. But few of themprovided empirical information on low effectiveness or harmful results of unstructured orinformal Off. Why should OJT be structured? Those authors used the terms are planned,organized, formal, and structured interchangeably. Although Rothwell (1991) used a term,"planned OJT" as equivalent to "structured OJT," the present author does not view that planned

OJT is necessarily structured. An unstructured OJT program can be planned consideringcharacteristics of the target tasks and economic value of the program. If a training program is

more structured, more money will be spent in developing and managing the program (Figure 2).It is unwise to use training budget in formalizing training for few trainees to obtain expertise

of simple tasks which are done infrequently.Conceptually, structured training can be differentiated from unstructured training, but

it should not be dichotomously judged: structured OJT is good, and unstructured one is bad.A more important issue is how much an Off program should be structured, not if it should be

18-4

EST COPY AVAILABLE

32

structure or unstructured. The issue is dependent on other variables: training unit (task, taskgroup, project, or job), number of trainees (small or many), and stability of training content(changing or stable).

Figure 2. Degree of Structure and Training Budget

trainingbudget

high

low

low high

Degree of Structure

R. A. Swanson (personal communication, January 25, 1995) developed a threedimensional model: the Training Delivery Method Model. The three variables are number oftrainees, content over time, and delivery method. Among them, the first and the second factorsare related to the question of structuredness. The more trainees need training and the stabletraining content over time is, the more formalized training, such as off-site classroom training,is appropriate (Figure 3 & 4). If the number of trainees is small and the target content ischanging, unstructured OJT is economic and appropriate.

Figure 3. Number of Trainees and Degree of Structure

high

low

degree ofstructure(formality)

small large

number of trainees

Besides economic considerations, characteristics of the tasks are another criterion usefulto judge the formality of training. If it is more important to know how individual tasks are tobe inter-related rather than individual tasks themselves, the training should be provided in a taskgroup or project as the training unit. Degree of structure correlates positively to size of trainingunits (Figure 5). For some tasks, especially at executive management level, which are beintertwined in a complex way, informal explanation on the job will be more understandable andprovide efficient learning. Instructional technologists should be aware ofstructuredness/formality issues before selecting or developing an OJT program.

18-4 33

Figure 4. Content over Time and Degree of Structure

high

degree ofstructure

low

changing stable

content over time

Figure 5. Training Unit and Degree of Structure

high

degree ofstructure

low

References

task project job

training unit

Anderson, J. R. (1985). Cognitive psychology and its implications. New York: W. H. Freemanand Company.

Anderson, J. R. (1987). Skill acquisition. Compilation of weak-method problem solutions.Psychological Review, 94 192-210.

Browthwell, M. M. (1995). The supervisor and On-the-Job Training (4th ed.). Reading, MA:Addison-Wesley.

Carnivale, A., & Gainer, L. (1989). The learning enterprise. Alexandria, VA: American Societyfor Training and Development and U.S. Department of Labor/Employment and TrainingAdministration.

Connor, J. C. (1983). On-the-iob Training. Boston: International Human ResourcesDevelopment.

Connor, J. (1988). One - to-one /Step -by -step. United Vergin Islands: TRC.DeJong, J. A. (1991). The multiple forms of on-site training. Human Resource

Development Quarterly, 2(4), 307-317.DeJong, J. A., & Versloot; B. (1994). Structuring on-the-job training. Paper presented at the

meeting of the 1st conference of the Academy of Human Resource Development; SanAntonio; TX. Engel, H. (1958). A practical guide to on-the-job training. SmallBusiness Bullitin No. 3, State of New York, Department of Commerce.

Gold, L. (1981). Four steps are no longer enough. Training and Development, 35(9), 28-32.

18-4

34

Goldstein, I. L. (1993). Training in organizations: Needs assessment, development, andevaluation (3rd ed.). Pacific Grove, CA: Brooks/Cole.

Jacobs, R. L. (1992). Structured on-the-job training. In H. D. Stolovitch, & E. J. Keeps (Eds.),Handbook of human resource performance technology (pp. 499-512). San Francisco:Jossy-Bass.

Jacobs, R. L. (1992). The multiple forms of on-site training: a rejoinder (HRDQ 2:4). HumanResource Development Quarterly, 3(1), 85-88.

Jacobs, R. L. (1992). Cross-training employees through structured OJT. Unpublished manuscript.Jacobs, R. L, & Jones, M. (1995). Structured on-the-iob training. San Francisco: Berrett-Koehler

Publishers.Kruger, M. J. (1985). Support that human resource development (HRD) managers provide to

first-line supervisors relevant to on-the-iob training (OJT). Unpublished doctoraldissertation, George Washington University, Washington, DC.

Lewick-Wallace, M., & Jask, R. C. (1988). Tips for effective on-the-job training, Performanceand Instruction, 27(7), 17-18.

Malcolm, S. E. (1992). Reengineering corporate training. Training, 29(8), 57-61.Mangum, S. L. (1985).On-the-job vs. classroom training: Some deciding factors. Training, 22(2),

75-77.Marquardt, M. J. (1976). The training function as perceived by supervisors on local departments

within the Virginia Department of Welfare. Unpublished doctoral dissertation; GeorgeWashington University.

Nadler, L. (1979). Developing human resources (2nd ed.). New York: Learning Concepts.Rackman, N. (1979). The coaching controvercy. Training and Development Journal, 33(11),

12-16.Rothwell, W. J. (1991). Invited reaction: Thoughts about on-the-job training and on-the-job

learning. Human Resource Development Quarterly, 2(4), 319-323.Rothwell, W. J., & Kazanas, H. C. (1990). Planned OJT is productive OJT. Training and

Development Journal, 44(10), 53-55.Rothwell, W. J., & Kazanas, H. C. (1990). Structured on-the-job training (SOJT) as perceived

by HRD professionals. Performance Improvement Quarterly, 3(3), 12-26.Senge, P. M. (1990). The fifth discipline. New York: Doubleday/Currency.Stokes, P. M. (1966). Total iob training: A manual for the working manager. : American

Management Association.Sullivan, R. F., & Miklas, D. C. (1985). On-the-job training that works. Training and

Development Journal, 39(5), 118-120.Wickman, M. A. (1989). On-the-job training: Formalizing informality or shouldn't supervisors

do the training? Performance & Instruction, 28(1), 31-32.

18-4 35

I agree to allow the editor of the 1995 Academy of Human ResourceDevelopment Proceedings, Elwood F. Holton III, to submit the

proceedings with my paper included to the ERIC database.

By signing this, I ar: releasing the paper for all authors of

the paper.

Print NameA5;17r6../4-7: cr r-r?

Singature e't e

Date

I agree to allow the editor of the 1995 Academy of Human ResourceDevelopment Proceedings, Elwood F. Holton III, to submit theproceedings with my paper included to the ERIC database.

By signing this, I am releasing the paper for all authors ofthe paper.

Print Name Li t- A IZ +1 (c--K

Singature

Date

I agree to allow the editor of the 1995 Academy of Human Resource

Development Proceedings, Elwood F. Holton III, to submit the

proceedings with my paper included to the ERIC database.

Ey signing this, I am releasing the paper for all authors of

the paper.

Print Name Sj at% et, mai , / irt

07/07/95 17145 a 025188703 WYATT KOREA

I agree to allow the editor of the 1995 Academy of Human ResourceDevelopment Proceedings, Elwood P. Belton I71, to submit theproceedings with my paper included to tie ERIC database.

By signing this, 7 am releasing the paper for all authors ofthe paper.

P.01

9

leaVA

Print Name3oN6r acta. Novir. 06,,,k,2..4-4 -.00)

Singature 0-12.4

Date 7/7 /IS--