Effectiveness of Motivationally AdaptiveComputer-Assisted Instruction on the DynamicAspects of Motivation

C] Song H Sc;-g'John M. Kelier

The purpose of this study was to examine theeWects of a prototype ofmotivationally-adaptive computer-assistedinstruction (CAI). The foundation iormotivational theory and design was providedby the ARCS model (an acronym formed fromattention, relevance, confidence, andsatisfaction). This model provides a definitionof motivation, a motivational design process,and recommendations for motivationalstrategies. Three treatment conditions wereconsidered: (a) mnotivationally adaptive CAI,(b) motivationally saturated CAI, and (c)motivationally minimized CAL Dependentvariables were eWectiveness, perceiVedmotivation (both overall motivation and eachof A, R, C, & S components), efficiency, andcontinuing motivation. The motivationallyadaptive CAI showed higher eWectiveness,overall motivation, and attention than theother two CAI types. For efficiencu, bothmotivationlally adaptive CAI andmotivationally minimized CAI were higherthan motivationally saturated CAI. Forcontinuing motivation, there were nosignificant differences among the three CAltypes, but a significant correlation wasfoundbetween overall motivation and continuingmotivation across the three CAI types. Thisstudy supports the conclusion that CAl can bedesigned to be motivationally adaptive torespond to changes in learner motivation thatmay occur over time. It also illustrates that theARCS model can be useful and effiective insupport of designing,for these dynamEic aspectsof motivation.

ETR&D, Vo'. 49, No. 2.2001, pp. 5-22 !SSN l042- 629

Computer-mediated instruction (Civi) is

being used ever more widely as an independent

deliverv system or to augment classroom and

Web-based learning environments. Learners in

both the school and the workplace will most

likely have increasingly greater amounts of

exposure to forms of CMI. However, despite the

convenience and potential effectiveness of CMI,

there continue to be problems with learner moti-

vation. Initially a new technology is appealing to

many people because of its novelty and the vari-

ety of features that add interest. However, as

computers are becoming more widely used for

instructional delivery, the motivation that

results from their novelty effects tends to disap-

pear (Keller, 1997; Keller & Suzuki, 1988). With

experience, students will no longer be as excited

by these novel features, and it then will becomemore of a challenge to stimulate and sustain

their motivation during computer-mediated

instruction.

One approach to making CMI more respon-

sive to individual student needs is by means of

adaptive instruction. However, several research-

ers in the area of adaptive computer-assisted

instruction (CAI) have raised concerns about

how to include adaptive responses to student

motivation. They have pointed out that fewr pre-

vious efforts to design adaptive CAI have con-

sidered leamer motivation (del Soldato & du

Boulay, 1995; Hativa & Lesgold, 1991;

McCombs, Eschenbrenner, & ONeill, 1973).

Their main concern has been that although pos-

sible benefits of developing adaptive CAI have

been found, one limitation of prior adaptive

instruction (e.g., Atkinson, 1976; Holland, 1977;

Houlihan, Finkelstein, & jonnson, 1992; Mills &

5

ETRi&D, Vo:, .49' No. 2

Ragan, 1994; Ross & Morrison, 1988; Tennyson& Christensen, 1988; Tennyson & Park, 1980) isthat student motivation to learn is disregardedor assumed to be embedded in the cognitivelyadaptive CAL. They have proposed the develop-ment of motivationally adaptive CAL that isadiusted to motivational changes for betterlearning outcomes. However, because none ofthe current approaches to CAI deals adequatelywith this problerm, the present study was con-ducted to dev elop and test an approach tomotivationally adaptive CAl based on a vali-dated approach to motivational design.

Current approaches to the design and devel-opment of motivating CAI can be divided intothree categories. The first is the computer-fea-ture approach, in which specific computer fea-tures and novelty effects (Clark, 1983) areassumed to increase the appeal of CAI, Forexample, Brown pointed out, "the student isrewarded by the use of the machine itself"(Brown, 1986, p. 28). Also, it has been contendedthat "the cumulative effects of entities such ascolor, graphics, and animation can beinstructionally and motivationally powerful"(Relan, 1992, p. 619). The second category is theprinciple-seeking approach, in which prescrip-tive motivational design principles and tacticsfor CAL are identified or developed from diversetheoretical and practical perspectives. For exam-pie, instructional games are recommended forthe development of motivating CAL k(Dempsey,Lucassen, Giluey, & Rasmussen, 1993; Malone,1981; Malouf, 1988). Cooperative learning onCAL (Johnson & Johnson, 1986), is also sug-gested, as is learner control of CAI (Klein & Kel-ler, 1990). The third category is themodel-establishing approach. The relationshipsbetween motivational theories and computerfeatures are identified and incorporated into apractical model for designing motivating CAI.For example, Lee and Boling (1996) proposedthe framework of motivational screen design,and Keller and Suzuki (1988) proposed a set ofmotivational strategies for designing motivatingcourseware.

Of the three approaches, the model-establish-ing approach appears to be the most useful inthat it provides a systematic and theoreticalframe of reference to guide instructional design-

ers. It could represent the dynamic nature ofmotivation and, therefore, help designers con-sider how learner motivation changes duringcomputer learning. A goal of tne present studvwas to ground it in such. an approach. Currentapproaches to motivational design in the contextof CAI are incomplete because they require thatall the motivational strategies be identified andpresequenced during the design and develop-ment phases, and then thev cannot be changedduring delivery. However, students who arehighly motivated before starting a CAI prograr,rwill not always remain motivated throughoutthe whole learning process. And conversely.some students who are not motivated at thebeginning may become motivated as they pro-ceed through the program. Therefore, it wouldbe highly desirable to enable the CAL package tobe responsive to motivational changes in thelearner at different times. In the present study!the idea of an adaptive provision of motivationalstrategies was tested on CAI as a way of provid-ing optimal motivational strategies.

One of the challenges in responding tochanges is to know when to delete motivationalstrategies as well as when to add them. If learn-ers are already motivated, thev may find itdemotivating to be exposed to unnecessaryenhancing motivational tactics (Astleitner &Keller, 1995; Keller, 1983, 1987 a, b, c). Therefore,a primary goal of motivationally adaptive CAi isto provide optimal motivational strategies tolearners, which means that the CAL should pro-vide the most appropriate motivational strate-gies in terms of purpose, type, and amount. Inother words, the computer should add appro-priate motivational strategies when the learnersare demotivated and remove unnecessary oneswhen they are highly motivated.

There have been a few pioneering efforts todevelop motivationally adaptive CAL (Astleitner& Keller, 1995; del Soldato & du Boulay, 1995;Rezabek, 1994). For example, Rezabek discussedthe use of intrinsic mnotivational strategies forthe development of a motivationaLlv adaptiveinstructional system, but did not provide anempirical test. Astleitner and Keller suggested asimulation approach for designing motivation-ally adapti.ve CAI in which they -mainly pro-vided ideas on how a computer can predict

6

7EFFECTIVENESS OF MOTIVATIONALLY ADAPTIVE CA:

motivational states. Del Soldato and du Boulay

developed an intelligent tutoring svstem in

which the system presented test items to be

solved by learners, and then both instructionalstrategies and motivational strategies were pre-

scribed based on student performance. How-

ever, none of these studies provided an adaptive

approach based on an assessment of iearners'

motivational states that can be emrbedded in a

variety of types of instructional strategies. It

would be desirable to conduct a studv that

investigates the possibility of developing

motivationally adaptive CAI for inclusion in

instruction that teaches diverse types of content

such as verbal information, concepts, and princi-

ples (Gagne, 1985). Regarding this need, del

Soldato & du Boulay suggested that "a richer

domain representation, including, for example,

a wider variety of links between topics, would

provide space for further elaboration of motiva-

tional tactics" (p. 373).

Another problem with the ideas suggested by

Astleitner and Keller (1995) and del Soldato and

du Boulay (1995) is that it is somewhat difficult

to apply their models to instructional design.

This is because their models basically try to

replace motivational capabilities of leamers

rather than responding to the learners' self-

reported motivational states. For example,Astleitner and Keller's approach requiresinstructional designers to obtain parameters,

such as incentive value and other mnotivational

constructs, to be used in their simulation. How-

ever, motivationally adaptive CAl doe.s not need

to be a complex intelligent tutoring system that

replaces leamer capabilities. It would be desir-

able to produce a generic. motivationally adap-

tive CAI that can be designed in a simpler and

more generalizable way.

Finally, it would be desirable to have a theo-

retical basis for the motivationally adaptive

approach that is based on an aggregation of re'l-

evant motivational concepts and theories, lends

itself to integration with instructional design,

and has been validated for effectiveness. In fact,

one possible reason that there has not been more

effort made to design a motivationally adaptive

CAI may have been the lack of a practical model

to guide the diagnosis and prescription of

learner motivational states (Astleimer & Keller,

1995). Although McCombs, Eschenbrenner, and

O'Neil drew attention to the need for designing

motivationally adaptive CAI in 1973, it was not

until recently that actual efforts were made to

develop such a system.

There have been many motivationa. theories

and models in educational psychology. For

example, behavioral theories explain motivationin terms of deprivation and reinforcement. The

cognitive view emphasizes attributional theoriesand intrinsic motivation arising from disequilib-

rium. The humanistic perspective focuses on

growth motivation or need gratification (see

Pintrich & Schunk, 1996, or any other basic text

on motivation for detailed explanation of these

and other approaches.) However, although

these perspectives may help instructionaldesigners understand motivation in. diverse

ways, they do not provide systematic guidelines

for instructional design.

Considering the above situation, the ARCS

model is useful for instructional designers

because it has the following features:

* It helps one understand the construct of moti-vation in terms of four distinct categories,

* It provides the systematic motivational

desigrn process, and

* It provides motivational strategies.

That is, the ARCS model combines a descriptivesynthesis of concepts and theories of motivation

with a systematic approach to motivational

design. It also provides motivational strategies

and tactics.

Historically, there have been other motiva-

tional design models. However, the early mod-

els tended to focus on one specific motivational

characteristic. One of the most prominent of

these is Alschuler's approach to developing the

achievement motive in children. He developed a

six-step process (Alschuler, 1973; Alschuler,

Tabor, & McIntyre, 1971), which was validated

and is still effective when applied appropriately.

The six-step process is used to arouse the

achievement motive in learners and then guide

them through a process of intemalization.

However, all of the applications of this type

of motivational design model fall umder the gen-

eral concept of psychological education. Their

purpose is to change the personality and behav-

ETR&DEV. Voi. 49, N5. 2

ior of students with respect to a specific motiva-tional characteristic rather than focusing moreholistically on the design of motivating learningenvironments.

In educational technology, there are twowell-published holistic models of motivationaldesign: (a) the time-continuum model ofWlodkowski (1999), and (b) Keller's ARCSmodel (1987a, b, c). The approach taken irn thisstudy is based on the ARCS model. The acronymARCS is derived from four categories of motiva-tional factors (attention, relevance, confidence,and satisfaction) that are based on an aggrega-tion of motivational concepts and theoriesaccording to their shared and discriminativeattributes. The ARCS model is grounded in ageneral macrotheorv of motivation and perfor-mance (Keller, 1979, 1983, 1999) and has beenvalidated in numerous studies and discussions(for example, Bickford, 1989; Keller, 1984; Klein& Freitag, 1992; Means, Jonassen, & Dwyer,1997; Newby, 1991; Nweagbara, 1993; Small &Gluck, 1994; Visser & Keller, 1990).

Wlodkowski's (1999) model provides a list ofcategories of motivational strategies and pre-scribes when to use them-at the beginning,during, or end of an episode of instruction. Thismodel is prescriptive in that it specifies whichtypes of strategies to use at prespecified timeintervals in the instructional process.

The ARCS model is similar to Wlodkowski'sapproach, but differs from, it in two importantways. (a) Strategy selection in the ARCS modelis done systematically from a set of categoriesand subcategories based on a comprehensivesynthesis of concepts and theories in humanmotivation. t(b The ARCS model is a problem-solving approach. Selection of strategies is basedon a systematic design process that includes ananalvsis of audience motivation. Strategies, boththe number and tape, are then chosen that areappropriate for the given audience.

This problem-solving approach of the ARCSmodel fits well into developing motivationallyadaptive CA] where learner analysis is a keychallenge, The categories of the ARCS modelprovide a basis for making CAl motivationallyadaptive by embedding motivational assess-ments at intervals in the lesson and then havingthe computer automatically include the most

appropriate motivational strategies in responseto each assessment.

In this study, a prototype of motivationallyadaptive CAI was developed which has the fol-lowing features:

* It contains a motivational analysis that wasdone frequently for attention, relevance, andconfidence. In a comprehensive approach toadaptive design, one would take measureson all four ARCS categories, and perhapseven some of the subcategories (Keller,1987c). especially in a long and comprehen-sive program of instruction. However, inshorter programns, attention, relevance, andconfidence are most important because thevare the factors that establish one's motiv ationto leam, and measures of them can easily beused for formative feedback. Measures of sat-isfaction would normally be taken after thelearners had finished a given block of instruc-tion hence they would be taken less fre-quently and would be more summative innature unless the program were long enoughto change the incentive structures and othersatisfaction elements. In the present studv anoutcome measure of satisfaction was takenalong with the other ARCS dimensions, butonly attention, relevance, and confidencewere included in the diagnostic and adaptivepart of the tutorial. It would be expected thatif a successful prototype were established,the process could be applied to longer pro-gramis and include satisfaction with the otherthree categories.

* The assessment method to be usedc for theembedded audience analysis was anotherconcern. Although Keller (1987b) contendedthat the full range of measurement possibili-ties ranging from self-assessment to perfor-mance observation and even electronicmeasures of physiological characteristics canbe considered in audience analvsis, the caseof CAI more or less limits one to self-assess-ment for practical reasons. In this study, thecomputer presented three questions at spe-cific internals in the program. The questionsasked learners to report their motivation withrespect to attention, relevance, and confi-dence. The researchers judged that thatwould be the most direct method and the eas-

8

EFFECTIVENESS OF MOTIVATIONA.LY ADAPTIVE CA'

iest to use for most instructional designers.However, as a way of partially compensatingfor the weakness of self-report measures,measures of performance scores on a quizwere also used in conjunction with the moti-vational self-assessment of confidence.

0 A distinction was drawn betvveen enhance-ment and sustaining tactics even though itwas difficult to make. If learners are alreadymotivated when beginning their study, theymay find it demotivating to be exposed tounnecessary enhancement motivational tac-tics (Astleitner & Keller, 1995. Keller, 1983,1987 a, b, c). For examnple, learners who areintrinsically motivated by a givern topic mayfind a tactic such as an unrelated "motiva-tional" game used as an. extrinsic reinforcer(as in Lepper, 1985) to be an annoying wasteof tirme. In these cases, just sustaining theirmotivation, not enhlancing it, is recom-mended. In contrast, in a lesson filled withdrill and practice exercises during a long dayat school, the opportunity to plav such agame could both be satisfying and help over-come boredom. Because it can be difficult toknow where to draw the line between thetwo categories of strategies, the learner moti-vational analysis is critical. It helps thedesigner determrine where enhancementstrategies are required to close motivationalgaps, and where there is simply the challengeof sustaining learner interest and confidence.

The purpose of this study was to test theeffectiveness, efficiency, and appeal ofmotivationally adaptive CAI that is based onself-report assessments of motivation with corn-puter-managed motivational strategies drawnfrom three key areas (attention, relevance, andconfidence) of motivation as described by theARCS model (Keller, 1987a). It was alsoexpected that the motivationa'Ily adaptive treat-ment would result in higher levels of self-reported interest in studying this topic in thefuture. This is in keeping with Maehr's (1976)concept of continuing motivation, although ituses self-report measures rather than observa-tions of future choices.

To achieve this purpose, a prototypemotivationally adaptive CAI was developedand compared to two different motivationally

prestructured versions of CAI. The first pre-structured version (motivationally saturatedCAI) was enhanced wuith a full set of bothenhancement and sustaining motivational strat-egies. In other words, the goal was to include alarge number of motivational tactics, as some-times happens when instructors or CAI design-ers incorporate many tactics without regard tothe actuaal motivational levels of the learners.Some researchers (for example, Farmer, 1989).found this to happen when the designers did notconduct a proper audience analysis. It wasassumed in the present study that an excessivenumber of tactics could annoy and demotivatelearners. The second prestructured version(motivationaily minimized CAI) contained aminimal number of tactics as required to main-tain good instructional design, but presumablynot enough to improve low motivation.

It was expected that motivationally adaptiveCAI would be most motivating to learnersbecause it was designed to: (a) avoid providingexcessive motivational strategies that would dis-tract or annoy learners who already found theinstruction to be motivating, and (b) overcomemotivational deficiencies among learners whowere bored or otherwise demotivated by thecontent. It was also expected that themotivationally adaptive CAI would be mosteffective in terms of learning achievement andsuccessful in producing the highest perceptionsof continuing motivation. Finally, the efficiencyof the three versions of CAI was determined bycomparing achievement to time spent on the les-son. It was expected that the motivationallyadaptive version would be most efficient.

METHOD

Participants and Design

Participants were 60 tenth-grade students fromthe Developmental Research School (DRS) affili-ated with a large university in Florida. Thesestudents were from three classes, two averageand one honors, taught by the same biologyteacher. Originally, 66 students were selectedfrom the approximately 85 students in theseclasses. Some were ineligible because they did

9

ETR&D. 'vd. 49. No. 2

not return parental approval forms or had some

familiarity with the content or software. Six stu-dents had to be eliminated from the final analy-

sis when, because of absences, their MAT 7

science scores were not available. Also, out of

the 60 students, only 59 were included. in. theanalysis for efficiency; one student scrarnbledthe program so that his data were lost.

Students are selected from the Leon CountySchool District and admitted to the DRS to be

representative of Florida's school-age popula-tiorn in terms of acadermic ability, race, sex, and

socio-economic status. The students in theclasses in this study represented the overalldiversitv in the school, except that there were no

participants from the lowest academic group or

from groups with special needs. The 22 originalparticipants in each of the three classes were

randormly assigned to the three levels of CAI(motivationally adaptive, motivationallv satu-

rated, and motivationally minimized). This ran-

dom assign-ment was expected to help control

for potential differences in the participants'entry level knowledge of genetics. The biologyteacher provided another control by confirming

that these students had not previously learnedthe content of genetics. The dependent variables

were effectiveness (learning), efficiency (amountof learning per time spent), motivation (both

overall motivation and each of attention. rele-vance, confidence, and satisfaction), and percep-

tion of continuing motivation.

materials

Three versions of HyperCard CAI were devel-

oped, containing identical contents on genetics.The contents were adopted and modified from

three sources: (a) a print-based biology instruc-

tional module developed by Osman (1992), (b) ahypertext CAI developed by Park (1993), and (c)

a textbook entitled Biology (Goodman et ai.,1989). Genetics was chosen for three reasons.

First, the instruction should have a range of con-

tent types such as facts, concepts, and principles.

This material, as developed by Osman and Park,

was systematicallv designed and validated

(Dick & Carev, T1996) and included those content

types. Second, the reading ability level was

appropriate to tenth grade students (Park, 1.993).Third, it was desirable that the content be some-what technical and not be easilv understood

without student effort, If the content were too

easy, it could be difficult to determine the effectsof motivational treatments and differences in

learning. It was confirmed by Park's study andby teachers in the current situation that thematerial had content validitv anid could be

expected to be sufficiently challenging.

Each of the three versions of CAI(motivationally saturated, adaptive, or mini-mized) was divided into three sections. At the

beginning of each section, the computer askedthe students three questions about their motiva-

tional attitudes. The first survey was presentedat the beginning of the lesson to deternine thestudents' initial attitudes toward the subject. In

each survey, they were asked about their level of

interest in the content ("1 find the subject ofgenetics interesting: Yes or No"), their percep-

tion of its usefuLness '"Learning genetics will be

useful to me: Yes or No'), and their confidencein learning it ("I feel confident that I can learngenetics: Yes or Nho").

Immediately following the motivationalanalvsis, they began studying their assigned

version and section of the instructional material,which was followed by a quiz covering that sec-

tion of content. At the end of the instruction,they were asked the continuing motivation

question, given the posttest, and given the sim-plified Instructional Material Motivation Survev

(1IMMS). Following is the sequence of events in

each of the three versions of CAl:

* Introduction and first embedded motiva-

tional analvsis

* First section of instructiorn and four-item quiz

o Second emnbedded motivational analysis

* Second section of instruction and four-item.

quiz

* Third embedded motivational analysis

* Third section of instruction

* Continuing motivation survey

* Posttest and simplified 11MMS

10

EFFECTIVENESS OF MOTIVATIONALLY ADAPTIVE CA

Treatments

The motivational strategies used in this studywere classified into the two categories of (a) sus-taining (Table 1) and (b) enhancing (Table 2)strategies. Keller (1983, 1987 a, b, c) distin-guished "enhancing motivation" from "'sustain-

ing motivation." He says that strategies formotivational enhancement are provided tolearners who require motivational improve-ments, while strategies for sustaining motiva-

tion are provided, as a kind of hygiene factor, toavoid having learners who show the desired lev-els of motivation become demotivated.

In the case of attention, Strategy AES2 (Table2), "engage the leamer's interest by using ques-tion-response-feedback interaction that recuiresactive thinking," and Strategy AES3, "presentproblem-solving situation in a context of explo-

ration and partial revelations of knowledge,"were categorized as enhancing strategies to usewith leamers whose motivation was lower than

desirable. As an example of AES2 during theexplanation of chromosomes and genes, a ques-tion ("Now, do you understand why the baby

has black hair?") was asked requiring learners tothink about the related concept. And then, anexplanation was provided. Also, as an exampleof AES3 when describing a picture of pea plants,

each part name was introduced, one by one, trig-gering learners' curiosity. However, StrategyASSI (TaDle 1), "keep instructional segments rel-atively short with progressive disclosure," wascategorized as an attention-sustaining strategy.For example, when introducing Mendel's life,

the text was presented sequentially in threesmall paragraphs instead of all at once in a large

paragraph.

For the motivationallyr minimized version,any motivational tactics embedded in Osman's(1992) text, Park's (1993) CAI, and the textbook

were elimninated except for certain sustainingtactics. For example, instructional tactics (suchas the inclusion of technical drawings in Figure1) that were judged by instructional designexperts as necessary for instructional effective-

ness were kept to maintain the inherent qualityof the instruction even if they might contribute

to positive motivation. This was to ensure thatthe developed CAI would have minimal motiva-

ting features for the students but still be

instructionallv effective. The motivationally

rinimized CAI included 3,017 expositorywords and 210 sentences with several insertedtechnical drawings. The Hypercard stack sizewas 112k, and it contained 59 cards. This version

of the lesson began with a title screen, an over-view of content, the motivational quiz, and an

Table E l' Motivation sustaining strategies.

Attention Sustaining Strategies (ASS)ASS1 Keep instructional segments relatively short with progressive disclosure.ASS2 Make effective use of screen display to facilitate ease of reading.ASS3 Intermingle information presentation screens with interactive screens.ASS4 Use a consistent screen format but with occasional variation.ASS5 Use visual enhancement functionallv to support the instruction and general theme of the lesson.ASS6 Avoid dysfunctional attention-getting effects such as a flashing word that distracts learner's

concentration.ASS7 Use underlines, italics, or bigger font sizes for the headings or key words.

Relevance Sustaining Strategies (RSS)RSS1 Use personal pronouns and the learners' name when appropriate.RSS2 Use graphic illustrations to embed abstract or unfamiliar concepts in a familiar setting.

Confidence Sustaining Strategies (CSS )CSS1 Allow the learner to escape and return to the menu at any time, and if feasible, to page backwards.CSS2 Give the learner control over pacing by hitting a key to go frorm one screen to the next.CSS3 Match learning requirements to prerequisite knowledge and skills to prevent excessive challenge or

boredom.

11

-ET&D. VoK 49 Nc 2

Table 2 Motvation enhancing strategies.

Strategies

Attention Enhancing Strategies (AES)AES' Use inverse and flash in text and patterns in pictures

as attention getters.AES2 Engage the learner's interest by using question-

response-feedback interaction that requiresactive thinking.

AES3 Present problem-solving situatior. in a context ofexploration and partial revelations of knowledge.

Relevance Enhancing Strategies (RE5)RESI Use examples from content areas and situation that

are familiar to the learnersRES2 Ciearlv state the objectives in terms of the

irmportance or utility of the lesson

ConFidence Enhancing Sfrategiet (CES)CESI U -se words and phrases that help attribute success

to the learners' effort and abilityCES2 Clearlv present the objectives and the overall

structure of the lesson.CES3 Explain the evaluative criteria and provide

opportunities for practice with feedback.CES4 Mention the prerequisite knowledge, ski'ls, or

attitudes that wi<l help the learner succeed atthe task.

CES5 Tell the learner how many items are going to bein a test or drill and whether it will be timed.

CES6 Provide summary.CES7 Use a menu-driven structure to provide learner

control over access to different part ofthe courseware.

Approrrnmate frequenc- of useSection 1 Section 2 Section 3

_ ~ ~ ~ ~ -- -.. . -

7

3

5

2

2

3

2

2 3

1

4 1

1

I 1

3

introduction. Screens were kept unclutteredwith parsimonious text (for example, as in Fig-ure 1) and illustrations were used where appro-priate to support learning.

The motivationally minimized CAI was not

expected to be highlv motivating to studentsexcept in t-o possible situations: (a) when the

content was intrinsically motivating to them, or(b) when the computer itself might have rnotiva-ting effects. However, these twvo situations werenot expected to affect many of the learnersbecause of the technical nature of the content. It

was also expected that there would be a rminimalnovelty effect of using the computer because ofthe previous cormputer-usage experiences ofleamers as confirmed bv their teachers. In anycase, an,y effects of these types were expected to

be randomnly distributed across treatments.Motivationally saturated CAl was developed

from the motivationallv minimized version. Tac-tic selection was based on the application o' theARCS model design process, motivational tac-tics presented by Keller and Suzuki (1988), tac-tics obtained frorn interviews with two biologyteachers, and other tactics proven in previousstudies (e.g., Bickford, 1989; Keller & Burkman,1993) to be motivating under appropriate cir-cumstances. These resouLrces provided a poolfrom which 24 were chosen that fulfilled theresults of audience analysis and rnet the follow-ing criteria established by Keller (1987b):

* Not take up too much instructional time,

* Not detract from the instructional objectives,

* Fall within. the time and money constraints of

12

4

II

I

I

13EFFECT!VENESS OF MOTIVAT IONALLY ADAPTIVE CAI

Figure 1 I_ Example of motivationally minimized screen.

As an individual formed by a sperm cell and an egg ceH grows into anadult, the genes influence its development. The genes cause it to resemblethe parents who suppHed the chromosomes. For example, the reason ababy has black hair is that its parents transferred genes for black hair tothe baby through chromosomes.

__1__ A pair of chromosomes

genes

NEXT

the development and

aspects of the instruction,implementation

* Be acceptable to the audience,

* Be compatible with the delivery system,

* Be appropriate for the contents, and

* Not bother multiple learners in the class-

room.

Because the motivationally saturated CAlincorporated all the motivational strategies

(both enhancing and sustaining strategies) forall three components (attention, relevance, and

confidence), it might seem that the motivation-

ally saturated CAI would maximize learner

motivation. However, research based on appli-

cation of the ARCS model (e.g., Farner, 1989;

Visser & Keller, 1990) did not support the

excessive use of motivational strategies because

it can actually decrease motivation bv annoying

the learner, which is the reason for including this

level of the independent variable. The

motivationally saturated CAI contained 3,306

expository words and 273 sentences. It also

included additional words (I p835, and sentences(247) for motivational strategies, with technical

drawings and tables inserted. The stack size was

393k, and it contained 107 cards.

In the saturated version, screens were either

added or modified in many instances to pro-

mote motivational features without adding new

content. For example, the screen in Figure 1 is

the 7th screen in the minimized program, but it

is the 12th in the sat-urated version. The addi-

tions included a personal orientation screen, twoscreens designed to enhance motivation, and

two "builds," which presented the material in

Figure 1 as a progressive disclosure. The content

of this screen was introduced as a problem by

asking the learner a question and then present-

ing the answTer in a later screen. Figure 2 illus-

trates a screen designed to stimulate curiosity

and relevance.

With motivationally adaptive CAI, a student

could encomnter up to 8 motivational scenarios

(combinations of motivational strategies) in the

first section of instruction, and up to 16 motiva-

tional enhancement scenarios in the second and

third sections. Prior to beginning the first sec-

tion, the student responded to the three-item

embedded motivational survey. There were six

possible combinations representing responses of

high or low to each of the three motivational

conditions (attention, relevance, and confi-

dence). Hence, there were three factors with two

* -

IF

ETR&D. Ve. 49, No. 2

Figure 2 = Sampie screen with curiosity and relevance tactics.

Think about the questions below for a momentbefore reading further. The answers will be in the reading material.

What do you think genetics is for?How do you think hair color, eye color, and facial structureare transferred from parents to offspring?

Mom's black Hair Dad's biack hair.

Baby's black hair

}o PRENUs11

levels each for a total of six. The rnotivationallysaturated group received all of the enhancingstrategies regardless of their responses, but stu-dents in the motivationallv adaptive groupreceived them only when they indicated lowattention, relevance, or confidence on any ofthree embedded motivational surveys. Forexample, a student w-ho checked "high" for curi-ositv and relev ance would not receive the screendisplayed in Figure 2.

At the end of the first and second sections ofinstruction, the results of the student's four-itermquiz and motivational survey were used for pre-scribing motivational strategies. The 8 types ofmotivational scenarios were elaborated into 16types based on the results (either high or low) ofthe confidence question in the four-iterr quizgiven at the end of each of those sections ofinstruction, The quiz was provided to check ifthe self-reported confidence level was consistentwith the performance scores on the quiz. A stu-dent who reported high confidence andobtained a high quiz score was considered tohave matched responses, but a student whoreported low confidence and had a high quiz

score was given a different type of feedback. Forexample, stLdents who reported low confidencebut obtained a high score received a reinforcingcomment for the high score and were told thatperhaps their confidence would grow strongernow, whrich could be considered to be a ty pe ofreattributional feedback. Twelve enhancementstrategies (Table 2) were usec; in various combi-nations in these scenarios.

instruments

Iotiv7atwn, Motivation was measured by a sim-plified version of Keller's (1993)1 IMMS, whichmeasures learner reactions to the motivationalfeatures of instructional material in terms ofattention, reievance, confidence, satisfaction,and overall motivation. Internal consistencsestimates for the IMMS total score and subscalesare generally in the range of .81 to .96.

The simplified INNS used for this study had16 items selected from Keller's original 36 items,with four items for each component of attention,relevance, confidence, and satisfaction respec-

14

i5EFFECTIVENESS OF MOTIVATIONALLV ADAPTIVE CAI

tively. Items were eliminated from selection that

did not reflect the motivational features of the

programs used. For example, the original item

"The exercises in this lesson were too difficult."

was eliminated because the CAI programs did

not have exercises. Students were asked to

respond on a five-point survey with response

choices of 1 (not true), 2 (slighltly true), 3 (moder-ately true), 4 (mostly true), and 5 (very true). The

potential total range score was from 16 to 80.

Sample items were, "The materials are eye-

catching" (A); "Completing this lesson success-

fully was important to me" (R); "I could

understand most of this lesson" (C;, and "I

really liked studying this lesson" (S). The reli-

ability coefficient obtained by Cronbach's Alpha

for the overall motivation measure was .92, and

it was .79, .73, .70, and .85 for attention, rele-

vance, confidence, and satisfaction, respectively.

Effectiveness. Learning achievement (effective-

ness) was measured by a I3-item posttest cover-

ing the following objectives: Students will be

able to:

* Choose correct features of pea piants that

enabled Mendel to succeed in discovering the

principles of heredityN.

* Select correct descriptions of Mendel's three

principles of heredity.

* Choose correct examples of concepts used in

the study of heredity.

* Solve hereditv problems using Mendel's

three principles.

The 13 multiple-choice items were adopted

or revised from a test with 24 itenms used in a

prior study (Park, 1993) so that students could

complete all the requirements within the experi-

mental time available for the study. Park

reported the reliability of the test as .78, using

the Kuder-Richardson formula (K-R 20). Items

were selected based on a table of specifications

for the content of instruction; redundant items

were excluded from selection. If necessarv, revi-

sions were made to clarify the meaning of the

questions. Two subject matter experts (biology

teachers) validated the test according to the

learning objectives using a table of specifications

for the content of instruction. Iternal consis-

tency of the test using the K-R 20 formula was

.69. The reduction in reliability compared to

Park's result could be due to the reduced length

of the posttest or the item selection.

Continuing motivation. Self-perception of con-

tinuing motivation (Maehr, 1976) was measured

by asking students if they wanted to learn more

in the future on the same or similar content. Stu-

dents were told to report their continuing moti-

vation on a Likert- type item with five anchors (1

= strongly disagree. 2 = disagree, 3 = not sure, 4 =

agree, 5 = strongly agree). One item stated, "The

lesson was motivating to me. I would like to

learn more about the same subject matter later."

E,ficiency. Efficiency was obtained by multiply-

ing 1,000 to the ratio of performance on the post-

test to study time. The computer tracked the

time (measured in seconds) used by each stu-

dent. For instance, if a student scored 8 using

2,000 seconds, 8 was divided by 2,000, and then

multiplied by 1,000 to yield an efficiency of 4.

Procedure

One month before the implementation of the

treatments, the students were measured on their

premotivation to leam genetics. This was doneto conduct audience motivational analysis as

recommended by the ARCS Model. The one-

way analysis of variance (ANOVA) reveaied no

significant difference amnong treatments for their

motivation to learn genetics, F(2, 57) = .40, MSE

= 13.60, p = .671. Also, a significant difference

was not found among groups on scores for

attention (F(2, 57) = .38, MSE = 1.90, p = .683), rel-

evance (F(2, 57)= .47, MSE =1.61, p = .630), con-

fidence (F(2, 57) = .67, MSE = 1.84, p = .515), and

satisfaction (F(2, 57) = .10, MSE = .30, p = .909).

The experiment was conducted for three

days at a leaming resource center (LRC) in a uni-

-versity building near the students' school. Each

day, students from one of the three participating

classes reported to the LRC. On the first day, the

22 students from the first class were randomly

assigned to the three treatments and completed

all of the materials. On the second and third

days, 22 students each from the other two partic-

ipating classes completed the same process.

ETR&D, VoL. 49. Nc. 2

Their biology teacher told all learners that theywould participate in an experiment, but theteacher did not -mention any credit for their par-ticipation. We wanted to observe how eachtreatmnent would enhance their motivation in theabsence of a strong extrinsic incentive. There-fore, although the topic of genetics was part oftheir regular curriculum, the teacher did not tryto force their desire to learn the contents. We rec-ommended that he not emphasize the irnpor-tance of participation. He simply arranged forthe students to participate in the experiment.

The experimenter (the first author) instructedthe participants to study the materials, empha-sizing Ethat a posttest and survey would be givenafterward. The computer assignment and expla-nation took about 10 to 15 minutes. Then, learn-ers were allowed to study as long as theywanted and the computer recorded their use oftime. They were also told to raise a hand whenthev were readv for the test. The average timeused by students was 30.49 min (range = 14.6-46.5 min). When students finished the lesson,their self-reported continuing motivation wasmeasured on the computer, which was followedby both the posttest and the simplified IMMS.

'Data Analysis

To test for differences in motivation and contin-uing motivation among learners in the three ver-sions of CA;, one-way ANOVA was used. Totest for differences in attention, relevance, confi-dence, and satisfaction, multivariate analysis ofvariance (MANOVA) was conducted. This anal-ysis was followed-up by appropriate univariateanalyses. To test for differences in effectiveness(learning achievement) and efficiency, one-wavanalysis of covariance (ANCOVA) was usedwith students' science scores on MetropolitanAchievement Test 7 as a covariate. ANCOVAwas conducted to control for the potential effectof prior achievement in science. Both ANOVAand ANCOVA were followed by Fisher's leastsignificant difference (LSD) pairwise compari-son procedure if a significant difference wasfound among treatments. Alpha was set at .05for all statistical tests. All these analyses weredone with the Statistical Package for the SocialScience (SPSS Windows V. 6.1).

RESULTS

Because of the attrition of six participants (seeParticipants and Design), there were 19 studentseach in the minimized and saturated versions ofthe CAI, and 22 in the adaptive version. Themeans and standard deviations for all depen-dent variables and conditions are presented inTable 3. Possible ranges for each variable arelisted in the footnotes to the table.

Motivation

The one-way ANOVA conducted on overallmotivation scores revealed a significant differ-ence for the treatments, FP2, 57) = 4.46, MSE =630.44, p < .05. Approximately 14% of the differ-ence in motivation was explained by the treat-ments. Observed power was .74. Fisher's LSDpairwise comparison procedures rexealed thatstudents in the motivationaliv adaptive CAT (M= 52.73, SD = 12.09) showed higher motivationthan those in both motivationally saturated (MA =

43.63, SD = 9.38) and motivationally rinimizedCAI (M = 42.84, SD = 13.75). The mean for themotivationaily adaptive CAI was at the 57thpercentile of the range bet-ween 16 and. 80,whereas those for the motivationally saturatedand mninimized versions were at the 43rd and42nd percentiles, respectiveiy.

Attention, Reievance, Confidence.Satisfaction

MANOVA indicated that the CAI treatmentshad an effect on components of motivation,Wilks's Lambda = .711, F(8, 1083 = 2.514, p < .05.U-nivariate analyses indicated that differencesoccurred only for attention and relevance. Forattention, there was a significant difference forthe treatments, F1P2, 57) = 5.07, MSE = 62.45, p <.01. Approximately 15% of the difference inattention was explained by the treatment, andobserxved power was .80. Fisher's LSD pairvisecomparison procedures revealed that studentsin the motivationally adaptive CAT (M = 13.36,SD = 3.36) showed higher attention than those inboth motivationally saturated (M = 10.95, SD =3.19) and motivationallv minimized CAI OM. =10.00, SD = 3.96). The mean for the motivation-

16

17EFFECT.VENESS OF MOTIVATIONALLY ADAPTIVE CAI

Table 3 Li Means and Standard Deviations for Dependent Variables

C.Al Versions

Devendent Variables Minimized Saturated Adaptiven 19 19 22

Overall Motivation1 M

SD

42.84

13.75

43.63

9.38

52.73

12.09

Attention2 M 10.00 10.95 13.36

SD 3.96 3.19 3.36

Relevance2 M 11.00 9.42 12.50

SD 3.64 2.97 3.47

Confidence2 M 11.95 12.00 13.95

SD 4.02 2.98 3.32

Satisfaction2 M 10.42 11.26 12.95

SD 4.22. 3.36 3.92

Effectiveness3 M 6.68 5.84 7.18(6.25) 6 (5.52) (7,94)

SD 3.35 3.11 2.50

Continuing Motivation4 A. 2.63 2.42 3.18

SD .96 1.07 1.10

Efficiency5 m

SD

Notes1: Possible range for overall motivation (16-K0).2: Possible range for attention, relevance, confidence. and satisfaction (4-20).3: Possible range for effecdiveness (0-131.4: Possible range for continuing motivation O -5).5: n = 18 for h4inimized Group.6: Adjusted scores for a covariate in parenthesis.

4.62(4,45)62.25

3.11(2.89)2.11

4.01(4.42)1.65

ally adaptive CAI was at the 59th percentile of the

range between 4 - 20, wheras those for the

motivationally saturated and minirmized versions

are at the 43rd and 38th percentiles, respectively.

Regarding relevance, there was a significantdifference for the treatments, F(2, 57) = 4.24,

MSE = 48.36, p < .05. Approximate.y 13% of thedifferences in relevance was explained by thetreatment, and observed power was .72, Fisher's

LSD pairwise comparison procedures revealed

that students in the motivationally adaptive CAI

(M = 12.50, SD = 3.47) showed higher relevance

than those in motivationally saturated CAl (M =

9,42, SD = 2.97). The mean for the motivationaIly

adaptive CAI was at the 53rd percentile of the

range between 4 and 20, whereas those for the

motivationally saturated and minimized ver-

sions were at the 34th and 44;th percentiles,

ETR&D. Vol. 49. No. 2

respectively. There was not a significant differ-ence for confidence and satisfaction.

Effectiveness

Because the correlation coefficient between sci-ence scores (covariate) and posttest scores wassignificantly high (r = .54, p = .001), one-way_ANCOVA was conducted on achievementscores. The results showed a significant differ-ence for the treatments, F(2, 56) = 5.28, MSE =29.58, p < .01. Approximately 16% of the differ-ence in acmievement was explained by the treat-ments. Observed powker was .82. Fisher's LSDpairwise comparison procedures revealed thatstudents in the motivationallv adaptive CAT(observed MA = 7.18, SD = 2.50) performed signif-icantly better than those in both motivationallvsaturated (observed M = 5.84, SD = 3.11) andmotivationally minimized CAI (observedM=6.68, SD=3.35). The overall achievement ofthe groups varied around the 50th percentilebecause of the difficulty level of the test. Thiswas deliberate in order to create a greater rangeof possible effects of the motivational tactics.

T'o investigate the validity of the rationale forthe assumption that increased motivation willproduce -more effectiveness, the correlationbetween overall motivation and adjusted post-test scores was calculated (there was no signifi-cant correlation between motivation andnonadjusted posttest scores). The observed post-test scores were transformed into adjustedscores for the covariate on the validated assump-tion of homogeneity of regression coefficient.The Pearsorn product-moment correlation r was.2546 (p < .05). This implies that about 6.5%o ofthe achievement variance can be explained bythe differences in overall motivation when theconfounding effects of the covariate are system-atically removed. This explained variance islarger than the 2.5%5 that was reported byBickford (1989), who used the ARCS model forprinted material instruction nonadaptively.

Continuing Motivation

Although the students in the motivationallyadaptive CAl (M = 3.18, SD = 1.0I) showedhigher self-reported continuing motivation than

both the motivationally saturated (M = 2.42, SD= 1.07) and motivationally minimized CAl (Al =2.63, SD = .96), the one-way ANOVA did notreveal a significant difference for the treatments,F(2, 57) = 2.93, MSE = 3.20, p = .061. Approxi-mately 9%. of the difference in continuing moti-vation was explained by the treatment.Observed power was .549.

Efficiency

As with effectiveness, the correlation coefficientbetween science scores and the efficiency mea-sure was significant (r = .4728, p = .001). One-way- ANCOVA revealedt a significant differencefor treatments, F(2, 55) = 5.17, MSE = 15.03, p <.01. Approximately 16, of the difference in effi-ciency was explained by the treatment, andobserved power was .81. Fisher's LSD pairwisecomparison procedures revealed that studentsin both the motivationally adaptive CAT(observed M = 4.01, SD = 1.65) and motivation-ally minimized CAI (observed AM = 4.62, SD =

2.25) showed higher efficiency than those in themotivationallv saturated CAT (observed M =

3.11, SD = 2. 11,

DISCUSSION

The purpose of this studv was to investigate theeffects of a motivationally adaptive CAT devel-oped in accordance with systematic motiva-tional design principles as represented in theARCS model (Keller, 1987a, b, 1999). For thispurpose, a prototype of motivationaily adaptiveCAI was developed and compared tomotivationaily saturated and motivationallvminimized versions for its effects on motivation(overall motivation, attention, relevance, confi-dence, and satisfaction), effectiveness, continu-ing motivation, and efficiency.

Results indicated that motivationally adap-tive CAI was superior to the other two CATtypes for the enhancement of overall motivationand attention. Regarding the other sub-components of motivation, the adaptive use ofmotivational strategies was partly supported.For relevance, students in the motivationailyadaptive CAI showAred higher scores than the

18

19ErFECTIVENESS OF MOTIVATIONALLY ADAPTIVE CAI

motivationally saturated CAI but not themotivationally mninimized version. Regardingconfidence and satisfaction, although studentsin the motivationally adaptive CAI showed the

highest scores, significant differences were notfound among three CAI types.

These findings may be explained as follows.The fact that students in the motivationally satu-rated CAI showed the lowest relevance scoresupports the contention that motivational strate-gies need to be prescribed optimallv; that is,adaptively. In the motivationallv saturated CAI,strategies for the three components of attention,relevance, and confidence were provided anddistributed throughout the instruction.. There-fore, students had to be exposed to much morematerial than those who received eithermotivationally adaptive or motivationally mini-

mized CAM. For example, the number of total

words used in the motivationaily saturated CAI

was 5,141, while the motivationallv minimizedCAI had only 3,017 words. Of the difference(2,124), 1,835 words were used for motivational

strategies, and this extra burden of words mayhave hindered the relevance strategies from

influencing students. Or, to put it differently, thelarge amount of irrelevant content may have

depressed their relevance scores.

The fact that there was no significant differ-ence for relevance between the motivationallymrinimized CAI and the motivationally adaptiveCAI was noteworthy. What would have pro-duced the unexpected relevance level in tfhemotivationally minimized CAL? One possibleexplanation is that student exposure to the con-tent orly, without distraction, allowed them torecognize the relevance as they learned moreabout the content even though they were notgiven specific relevance strategies. This infer-ence may be supported by the fact that studentsin the motivationally minimized CAI showedhigher achievement than those in themotivationally saturated CAi who were pre-sumed to be distracted by all the motivational

strategies. However, the difference in achieve-ment between these two treatments was not sig-nificant, so additional research would have to bedone to confirm or negate this inference.

Another interesting finding is that there wereno differences in confidence among the three

conditions. This can be interpreted in two ways.First, it is possible that the prescribed confidencestrategies (such as "use words and phrases thathelp attribute success to the learners' effort andability," shown in Table 2) were not strongenough to address the motivational needs. Infact, this is a possible inference in that not all ofthe powerful computer features could be usedbecause of the constraints of experimental con-ditions. Second, provision, of review opportuni-ties could have resulted in nonsignificantdifferences in confidence among the three condi-tions. Research results show that review oppor-tunities are effective strategies for increasingstudent confidence (Bickford,1989). Therefore,review opportunities that were provided beforeand after each quiz may have contributed tomaintaining or enihancing learner confidenceregardless of condition. Further investigation ofthis inference needs to be done in future studies.

Aside from the motivational benefits of theadaptive use of the ARCS model, anotherexpected benefit was to increase achievementscores. Although motivation to learn is not theonly predictor of student achievement, itseemed reasonable to predict that if motivation-ally adaptive CAI is more motivating than theother two CAI types, it would also show7 thehighest effectiveness. The results of this studysupported this expectation.

Regarding continuing motivation, which wasself-reported in this study, Keller (1983, 1987a, b)pointed out that the rnore highly motivated thelearners are during their learning, the betterchance of their having high continuing motiva-tion after their learning. In the present study,students in the motivationally adaptive CAIshowed the highest continuing motivation, but

it was not enough to be significantly differentfrom those of students in the other two CAI

types. The observed power for ANOVA on con-tinuing motivation was .549; therefore it might

be that a larger sample size or longer treatmentwould result in significant differences among

the three conditions. If one considers the rela-tionship between overall motivation and contin-uing motivation in the three conditions, the

correlations were in the right direction. Themotivationally adaptive CAI showed the high-

est correlation (r = .82, p < .001), while the

E`P&D, Vol 49. No. 2

motivationally minimized CAl was also signifi-cant (r = .64, p = .003). It is interesting that themotivationally saturated CAI showed the lowestcorrelation, r = .34, which was not significant (p= .151).

For efficiency, it was expected that learners inthe motivationally adaptive CAI would needless time to finish their learning than those whohad to go through all the motivational strategiesin the motivationally saturated CAL. Also, it wasexpected that the motivationally minimzed CALwould require less time to finish thfan, the otherconditions because no motivation enhancementstrategies were provided. The analyses of effi-ciency showed that the motivationally adaptiveand motivationally minimized conditions hadhigher efficiency than the motivationallv satu-rated condition. These findings support the needfor providing motivational strategies adaptivelyfor efficiency. The excessive provision of motiva-tional strategies in the motivationally saturatedCAI resulted, as expected, in lower efficiencvthan the motivationally mrinimized CAI, whilethe motivationally adaptive CAI showed thesame efficiencv with the motivationaliv mini-mized CAL.

In summary, motbvationaliy adaptive CAI.was generally supported by the data in terms ofits motivation, effectiveness, overall motivation,and efficiency. The data also indicated that thereis a significant correlation between motivationand continuing motivation. These findings sup-port the adaptive provision of motivationalstrategies in CAL.

Oonsiderotbons for Future Research

As an initial attempt to desigrn and develop amotivationallv adaptive CAT, this study has lim-itations to be considered in the design of futurestudies. First, the potential range of motivationalstrategies could not be fully actualized becauseof constraints of availabie hardware and soft-ware. For example, after instruction, many stu-dents commented on the lack of color,anirmation, and sound for more motivationalfeatures. Some important conditions (Astleitner& Keller, 1995) for motivated learners, such asself-created task solution, opportunities for

advanced topics, and control over importantparts of the learning environment were notused.

Second, the time for the experimnent wasshort. The average time was 30.49 minutesacross three conditions (motivationallv inii-mized CAI: 24.37, saturated CAI: 35.45, adaptiveCAL: 31.21). Considering the limited use of moti-vational strategies as discussed above, thisshort-term exposure to motivational strategiescould have been another reason for showingseveral insignificant differences despite theoverall positive results that were found.

Third, the use of self-report methods for mea-suring motivation was lirnited in that suchmethods required students to indicate their per-ceived motivation level, which rright have beendifferent from their actual amount of effort-amore accurate measure of motivational behav-ior. Also, the embedded motivational analysesmay have been intrusive, requiring students tostop their process of learning. A more naturalway of measuring motivation would be desir-able.

Fourth, motivational strategies used in thisstudy may not have been as closely targeted tothe real motivational needs of students as wouL:ldbe desirable. The present study used one globalquestion for each of the three major componentsof motivation. It might be more effective to focuson the subcategories of each component formore reliable diagnosis and prescription. Forexample, confidence is composed of the sub-components of motive matching, goal orienta-tion, and familiaritv (Keller, 1937a, c). Ifmotivational analysis were done for each ofthese subcomponents, adaptive CAI might bemore effective in enhancing student confidence.

Conclusion

Despite the limitations in this prototype devel-opment studv, the results demonstrate that it isfeasible to design motivationally adaptiveinstruction for a self-paced learniing setting suchas CAI. Thev also demonstrate that self-reportsof motivation can be valid indicators of learningreadiness to which motivational strategies areadaptively prescribed. And, the study

20

21EFFECTIVENESS OF MOTIV.TIONALLY ADAPTIVE CAi

demonstrates that the ARCS model can be

applied effectively to the design of motivation-

ally adaptive CAL. Further research should lead

to more sophisticated and effective applications

of motivationally adaptive design. F

Sang H. Song is at Andong National Universitqy,Korea.

John M. Keller is at Florida State. University.

REFERENCES

Alschuler, A.S. (1973). Developing achievement motiva-tion in adolescents: Education for human growth Engle-wood Cliffs, N7: Educational TechnologyPublications.

Alschuler, A.S., Tabor, D., & McIntyre, T. (1971). Teach-

ing achiezement motivation: Theory and: practice in psy-chological education. Middletown, CT: EducationVentures, Inc.

Astleitner, J., & Keller, J.M. (19951. A model formotivationally adaptive computer-assisted instruc-tion. Joumnal of Research on Computing in Education,27(31, 270-280.

Atkinson, R.C. (1976). Adaptive instructional systems:Some attempts to optimize the learning process. InD. Klahr (Ed.), Cognition and instruction (pp. 8I-108).New York: Wiley & Sons.

Bickford, N.L. (1989). The systematic application of prin-

ciples of motivation to the design of printed instructionalmaterials. Unpublished doctoral dissertation. FloridaState Universitv, Florida.

Brown, J. (1986). Sorne motivational issues in com-

puter-based instruction. Educational Technology.26(4), 27-29.

Clark, R.E. (1983). Reconsidering research on learningfrom media. Review of Educational Research, 53(4),

445-459.del Soldato, T., & du Boulay, B. (1995). Implementation

of motivational tactics in tutoring systems. Journal of

Artificial Intelligence in Education, 6(4), 337-378.Dempsey, J., Lucassen, B., Gillev, W., & Rasmussen, K.

(1993). Since Malone's theorv of intrinsicallv motiva-ting instruction: What's the score in the gaming liter-ature? journal of Educational Technology Systems,22(2), 1,73-184.

Dick, W., & Carey, L. (1996). The systematic design ofinstruction (4th ed.). New York: Harper Collins.

Farmer, T.M. (1989). A refinement of the ARCS motiva-tional design procedure using a formati've evaluationmethodology. Unpublished doctoral dissertation,Indiana University, Indiana.

Gagne, R.M. (1985). The conditiorns of learning and theoryof instruction (4th ed.). New York: Hilt, Rinehart &Winston.

Goodman, H.D., et al. (1989). Biology. Orlando, FL:Harcourt Brace Jovanovich.

I-lativa, N., & Lesgold, A. (1991). The computer as atutor-Can it adapt to the individual learner?

Instructional Science, 20(l), 49-78.Holland, J .G. (1977). Variables in adaptive decisions in

individual instruction. Educational Psychologist,

12(2),146-161.Houllihan, P.A., Finkelstein, M.W., & Johnson, L.A.

(1992). Adaptive use of the DDx&Tx system. Journalof Computer-Based Instruction. 19(4). 125-130.

Johnson, D.W., & Johnson, R.T. (1986). Computer-assisted cooperative learning. Educational Technol-ogy, 26(1), 12-18.

Keller, j.M. (1979). Motivation and instructioraldesign: A theoretical perspective. Journal of Instruc-tional Development, 2(4), 26-34.

Keller, l.M. (1983). Motivational design of instruction.In C.M. Reigeluth (Ed.), Instructional-design theoriesand models: An overview of their current status (pp.386-434). Hillsdale, NJ: Lawrence Erlbaum Associ-ates.

Keller, J.M. (1987a). Strategies for stimulating the moti-vation to learn. Performance and Instruction Journal,

26(8), 1-7.Keller, J.M. (1987b). The systematic process of motiva-

tional design. Performance and Instruction Journal,26(9 /10),1-8.

Keller, J.M. (1987c). Development and use of the ARCSmodel of instructional design. Journal of InstructionalDevelopment. 10(3), 2-10.

Keller, J.M. (1993). Motivation by design. Unpublishedmanuscript, Florida State University, Florida.

Keller, '.M. (1997). Motivational design and multime-dia: Beyond the novelty effect. Strategic HumanResource Development Review, 1(1), 188-203.

Keller, j.M. (1999). Motivational svstems. In H.Stolovitch, & E. Keeps (Eds.), Handbook of human per-

formance technology (2nd ed.). San Francisco: Tossey-Bass Inc. Publishers.

Keller, J.M., & Burkmaen, E. (1993). Motivation princi-ples. In M. Fleming & W.H. Levie (Eds.), Instruc-tional message design: Principles,from the behavioral andcognitive sciences. Englewood Cliffs, NJ: EducationalTechnology Press.

Keller, J.M., & Suzuki, K. (1988). Use of the ARCS moti-vation model in courseware design. Lin D.H.Jonassen (Ed.), Instructional designs for microcomputercourseware (pp. 401-434. Hillsdale, NJ: LawrenceErlbaum Associates.

Klein, J.D., & Freitag, E.T. (1992). Training students toutilize self-motivational strategies. Educational Tech-noiogy, 32(3), 44-48.

Klein, J.D., & Keller, J.M. (1990). Influence of studentability, locus of control, and type of instructionalcontrol on performance and confidence. Journal ofEducational Research, 83(3),140-146.

Lee, S.H. & Boling, E. (1996). Motivationai screen design

guidelines for effective computer-mediated instruction.Proceedings of Selected Research and DeveloprnentPresentations at the 1996 National Convention of the

Association for Educational Cormnmunications and

ETR&D_. VO.. 49. Nc. 2

Technology, Indianapolis, Indiana, 401-412.Lepper, M.R (1985). Microcomputers in education.

Motivational and social issues, American Psycholo-gist, 40(l), 1-18.

Maehr, M.L. (1976). Continuing motivation: An analy-sis o. a seldomn considered educational outcome.Review of Educational Research, 46(3). 443-462.

Malone, T. (1981). Toward a theorv of intrinsicallymotivation instruction, Cognitive Science, 4. 333-369.

Malouf, D.B. (1988). The effect of instructional corn-puter games on continuing student motivation. Thelournai of Special Educatior, 21(4), 27-38.

McCombs, B.L., Eschenbrenner, A.J., Jr., & O'Neill,H.F., Jr. (t973). An adaptive model for utilizinglearner characteristics in computer based instruc-tional svstems. Educational Technology, 13(4), 47-51.

Means, T.B.. fonassen, D.H., & Dwyer, F.M. (1997).Enhancing relevance: Embedded ARCS strategiesvs. purpose. Educational Technology Research andDevelopment, 451), 5-18.

Milis, S.C., & Ragan, T.J. (1994). Adapting instruction tcindividual iearner di ,erences: A research paradigm forconmputer-based instruction. Proceedings of SeiectedResearch and Development Presentations at the1994 National Convention of the Association forEducational Comrnm-unications and Technology,Nashville, Tennessee, 525-546.

Newby, T.J. (1991), Classroomrn motivation: Strategiesof first-year teachers. Journal of Educational Psvchol-ogy, 83 (2, 195-200.

Nwagbara, CI. (19931. Effects of the relevance componentofthe ARCS model of motivzational design. Unpublisheddoctoral dissertation. Purdue University, Indiana.

Osman, M.- "992). The etfects ofthink-ahead questions andprior kniurledge on learning and retention. Unpub-lished Doctoral Dissertation, Florida State Un;ver-sitv, Florida.

Park, IW. (1993). The cffects of orienting questions andprior knowledge on? learning in hypertext. ULnpublisheddoctoral dissertation, Florida State Un-iversity, Flor-ida.

Pintrich. P.R., & Schunk, D.fH. 11996). Motivation in edu-cation: Theory, research, and applications. Englewood,Cliffs, NJ: Prentice Hall,

Relan, A. (1992). Motiv'ational strategies in computer-based instruction: Some lessons from theories and modelso,f motivation. Proceedings of Selected Research andDevelopment Presentations at the Convention of theAssociation for Educational Communications andTechnology, 612-624.

Rezabek. R.H. (1994). Utilizing intrinsic motivation in thedesign of instruction. Proceedings of SelectedResearch and Development Presentations at theNational Convention of the Association for Educa-tional Communications and Technology, Nashville,Indiana, 695-704.

Ross, S.M., & Morrison, C.R. (1988). Adapting instruc-tion to learner performance and background vari-ables. ln D.H. Jonassen (Ed.), Instructional designs formicrocomputer courseware (pp. 227-243). Hillsdale,N1: Lawrence Erlbaum Associates.

Small, RV., & Gluck, M. (19941. The relationship ofmotivational conditions to effective instructionalattributes: A magnitude scaling approach. Educa-tional Technoiogy, 34(8), 33-40.

Tennyson, R.D., & Christensen, D.L. (1988). MAIS: anintelligent learning system. In D. Jonassen, (Ed.),Irnstructional designs for micro computer courseioare(pp. 247-74). -lillsdale, NJ: Lawrence Erlbaurn.

Tenrnyson, R.D.. & Park, S. (1984). Process learningtime as an adaptive desigrn variable in concept learn-ing using computer-based instruction. Journal oFEducational Psycholog 6., 452-465.

Tuckman, B.W. (1994). Conducting educational research(4th ed.). New York: Harcourt Brace & Company.

Visser, J., & Kelier, J.M (1990). The clinical use of moti-vational messages: An inquirv into the validitv ofthe ARCS model o.' motivational design. Instruc-tional Science. 19. 467-500.

Wlodkowski, R J. (1999). Enhancing adult motivation tolearn (2nd ed.,). San Francisco: Jossey-Bass

22

COPYRIGHT INFORMATION

TITLE: Effectiveness of motivationally adaptive computer-assistedinstructionon the dynamic aspects of motivation

SOURCE: Educational Technology Research and Development 49no2 2001

WN: 0100203447001

The magazine publisher is the copyright holder of this article and itis reproduced with permission. Further reproduction of this article inviolation of the copyright is prohibited.

Copyright 1982-2001 The H.W. Wilson Company. All rights reserved.