review of research in revising instructional text

7/30/2019 Review of research in revising instructional text

1/28

http://jlr.sagepub.com/Journal of Literacy Research

http://jlr.sagepub.com/content/23/3/307The online version of this article can be found at:

DOI: 10.1080/10862969109547744

1991 23: 307Journal of Literacy ResearchMary H. Sawyer

A Review of Research in Revising Instructional Text

Published by:

http://www.sagepublications.com

On behalf of:

Literary Research Association

can be found at:Journal of Literacy ResearchAdditional services and information for

http://jlr.sagepub.com/cgi/alertsEmail Alerts:

http://jlr.sagepub.com/subscriptionsSubscriptions:

http://www.sagepub.com/journalsReprints.navReprints:

http://www.sagepub.com/journalsPermissions.navPermissions:

http://jlr.sagepub.com/content/23/3/307.refs.htmlCitations:

What is This?

- Sep 1, 1991Version of Record>>

by guest on May 1, 2013jlr.sagepub.comDownloaded from
http://jlr.sagepub.com/http://jlr.sagepub.com/http://jlr.sagepub.com/content/23/3/307http://jlr.sagepub.com/content/23/3/307http://www.sagepublications.com/http://www.sagepublications.com/http://www.nrconline.org/http://jlr.sagepub.com/cgi/alertshttp://jlr.sagepub.com/cgi/alertshttp://jlr.sagepub.com/subscriptionshttp://jlr.sagepub.com/subscriptionshttp://jlr.sagepub.com/subscriptionshttp://www.sagepub.com/journalsReprints.navhttp://www.sagepub.com/journalsReprints.navhttp://www.sagepub.com/journalsPermissions.navhttp://www.sagepub.com/journalsPermissions.navhttp://www.sagepub.com/journalsPermissions.navhttp://jlr.sagepub.com/content/23/3/307.refs.htmlhttp://online.sagepub.com/site/sphelp/vorhelp.xhtmlhttp://online.sagepub.com/site/sphelp/vorhelp.xhtmlhttp://jlr.sagepub.com/content/23/3/307.full.pdfhttp://jlr.sagepub.com/content/23/3/307.full.pdfhttp://jlr.sagepub.com/http://jlr.sagepub.com/http://jlr.sagepub.com/http://online.sagepub.com/site/sphelp/vorhelp.xhtmlhttp://jlr.sagepub.com/content/23/3/307.full.pdfhttp://jlr.sagepub.com/content/23/3/307.refs.htmlhttp://www.sagepub.com/journalsPermissions.navhttp://www.sagepub.com/journalsReprints.navhttp://jlr.sagepub.com/subscriptionshttp://jlr.sagepub.com/cgi/alertshttp://www.nrconline.org/http://www.sagepublications.com/http://jlr.sagepub.com/content/23/3/307http://jlr.sagepub.com/


2/28

Journal of Reading Behavior1991, Volume XXIII, No. 3

A REVIEW OF RESEARCH IN REVISING INSTRUCTIONALTEXT

Mary H. SawyerState University of New York at Albany

ABSTRACTSchool textbooks are commonly criticized. This article reviews research in revisinginstructional text by examining multidisciplinary research in readability, textstructure, text interestingness, expert revisers' strategies, and readers' compre-hension strategies. Findings are that much of this research is limited by a simplisticview of reading, the use of experimentally contrived texts and contexts, and adependency on recall as a measure of comprehension. A promising line of recentresearch uses readers' verbal protocols to guide revision and develop revisionprinciples. Further research needs to: (a) develop other innovative ways ofmeasuring comprehension, (b) focus on subject specific concerns, and (c) examineclassroom contexts of textbook use.Central to the school curriculum are textbooks; in many cases they determineboth what is taught and how it is taught (Elliott & Woodward, 1990). They arethe primary learning tool given to studentsresearchers have found that 98% ofclassroom instruction is from materials, not teachers, and 90% of students' home-work time is structured by materials (see Jackson, 1981). Further evidence of thecentrality of textbooks in the curriculum is seen in the controversies surroundinghow the theories of evolution and creation are treated in science textbooks or howwomen and minorities are depicted in social studies textbooks. In some cases

textbooks truly are students' sole source of information, such as in independentstudy and correspondence courses.Yet textbooks have come under criticism for being "narrow and limiting"(Applebee, 1984, p. 35), "inconsiderate" (Armbruster, 1988, p. 3), and "choppy,stilted, and m onotonous" (Tyson-Bernstein, 1988, p. 21 ). Anderson and Armbrus-ter (1984) have argued that textbooks lack coherence and completeness whereas

307by guest on May 1, 2013jlr.sagepub.comDownloaded from
http://jlr.sagepub.com/http://jlr.sagepub.com/http://jlr.sagepub.com/http://jlr.sagepub.com/


3/28

308 Journal of Reading Behavior

Anderson, Shirey, Wilson, and Fielding (1986) have questioned textbooks' ability"to arouse and hold a child's interest" (p. 298).This review looks at research relevant to improving instructional prose. Theterm instructional prose is used in order to distinguish it from the expositoryprose one might find in magazine or newspaper writing. The primary purpose ofinstructional prose is to teach. Because improving instructional writing is not adiscipline specific concern, research comes from a variety of areas: cognitive psy-chology, reading, composition, rhetoric, and linguistics. The motivations for theresearch also vary widely. For example, there are those who wish to improvetechnical manuals so army engineers can repair sophisticated equipment, those whowant to write physics texts in a way that will help students understand difficultconcepts, and those who wish to make social studies texts more memorable andexciting. Because the focus is on expository instructional text, research in compre-hending and improving narrative text is largely irrelevant and is thus, for the mostpart, excluded from this review.

This review is divided into five sections. The first section briefly examinesresearch in readability. The second section focuses on the greatest body ofresearchthat which examines the effects of text structure. Also included in thissection are findings from adjunct aid and advance organizer research. The thirdsection looks at a relatively recent strand of research that looks at the effects oftext interestingness. The fourth section looks at research on expert revisers' strate-gies and their effects on reading com prehension. The research reviewed in the fifthsection departs from traditional methodologies. Here, the focus is on studying howread ers' com prehension strategies can guide text revision. Although no article-sizedliterature review of these five research areas could really be complete, an attempthas been made to document and make sense of each of their major concerns andfindings.

READABILITYReadability research, usually said to begin with Thorndike's (1921) publicationof word frequency counts (Britton, Van Dusen, Gulgoz, & Glynn, 1989; Clifford,1978; Moore, Readence, & Rickelman, 1983), marks the earliest effort to providerevision guidelines for instructional text. This research is concerned with providing

quantitative measures of writing (readability formulas) to determine its appropriate-ness for various grade and ability levels. The various formulas (e.g., Coleman &Liau, 1975; Flesch, 1948; Kincaid, Fishburne, Rogers, & Chissom, 1975) deter-mine a grade level for a text by essentially measuring the length of sentences andthe familiarity of the vocabulary. Reviews of readability research (Kern, 1980;Klare, 1984; Red ish, 1979 ), how ever, indicate their limited use in guiding textbookrevision beyond the sentence level. Though these formulas are the most widely



4/28

Revising Instructional Text 309

used guidelines in textbook revision, they are also widely attacked for being mis-leading and overly simplistic (Charrow, 1979; Davidson & Green, 1988; Duffy,1985; Duffy & Kaba nce , 1982; Ho lland, 1981; Klare, 1963, 1976, 1984; Red ish,1979).In his review of readability research, Klare (1984) identified two ways ofthinking about the validity of reading formulas: the prediction of readable writingand the production of readable writing. For our purpose of improving instructionaltext, we might ask: (a) Do readability formulas accurately predict more understand-able text for intended readers? and, (b) Can readability formulas serve as effectiveguides for producing more understandable writing for intended readers? In responseto the first question, because readability formulas omit important variables, theylack both face and predictive validity. Coupland (cited in Klare, 1984) wrote that"t he simplicity of . . . readability formulas . . . does not seem compatible withthe extreme complexity of what is being assessed" (p. 15). Redish (1979) notedthat readability formulas do not measure the complexity or logic of ideas, andHolland (1981) found readability formulas insufficient for evaluating the readabilityof public documents (such as forms, brochures, and regulations). Holland (1981)listed crucial variables not accounted for in the readability formulas: (a) otherword and sentence variables, such as word concreteness and sentence syntax; (b)contextual factors, such as text structure and cohesion; the content and readers'prior knowledge; the function of a document and readers' corresponding goals; atext's graphs, tables, headings, type face, color, indentation, blocking and whitespace; reader characteristics, such as their culturally-based expectations. She con-cluded:

Thus, over and above readability variables, a range of interacting factors influencehow well a document will be understood. . . . While sentence length and wordfrequency do contribute to the difficulty of a document, a number of equallyimportant variables elude and sometimes run at cross purposes to the formulas. . . .(Holland, 1981, p. 15)

Since these critical factors are not taken into account in the formulas, the Interna-tional Reading Association and the National Council of Teachers of English issueda joint statement declaring the wide misuse of readability formulas saying, "It isnot what readability formulae measure that concerns us; it is what they do NOTmeasure" (International Reading Association, 1984/1985). Klare (1984) alsodiscussed the questionable validity of formulas which yield scores on a grade-placement scale.

Thus, it should not suprise us to find that when materials are revised to makethem more "readable," gains in comprehension are often either nonexistent orinsignificant (Britton et al., 1989; Duffy & Kabance, 1982; Johnson & Otto, 1982;Klare, 1976; Kniffen, Stevenson, Klare, Entin, & Slaughter, 1979; Walmsley,Scott, & Lehrer, 1981). Duffy and Kabance (1982) examined a readable writing



5/28


approach to text revision and found that a reduction of six grade levels failed toproduce practical improvements in comprehension on several reading-to-do tasks.More readable texts (as defined by formulas) can actually be more difficult tounderstand. Pearson (1975) found that grammatical complexity was often an aid tochildren's comprehension and recall rather than a hindrance. Furthermore, Bruner(1984) argued that text oversimplification in basal readers can produce childrenwith so-called reading difficulties.

As far as whether readability formulas can serve as effective guides for writingmore readable texts, the answer is resoundingly no. Klare (1984) advised writersto "keep formulas out of the writing process itself" (p. 730) and called the p racticeof writing to formula "ineffective if not dangerous" (p. 712). Redish (1979) em-phasized that because readability formulas are correlational, "the features that arecounted are only an index of the problem, not an explanation" (p. 7) and, for thisreason, the formulas should not be used as guides for rewriting.

Clearly reading formulas as guidelines to improving instructional writing areinadequate, yet they remain popular because they are easy to use, inexpensive, andprovide a replicable quantitative index. As Redish (1979) explained, "Readabilityformulas serve a purpose for which no other handy tool now exists. They allowthe writer to review work and get a quick idea of relative reading level withouthaving to leave the office" (p . 6). Yet their inadequacies force researchers to searchfor, if not alternatives, then additional methods that can help identify appropriateand readable instructional text and can serve as guidelines for instructional textrevisers. We should think of readability formulas as helpful screening devices(Klare, 1984) and readability level " a s a filter . . . to decide which texts are notacceptab le" (Holland, 1981 , p . 15).

Readability research also does not address critical functions of texts in instruc-tional contexts. For example, the merits of collaborative learning on helping de-velop students' critical thinking has been documented (Abercrombie, 1960; Barnes,1976; Miller, 1988; Sweigart, in press), yet instructional texts are not analyzed fortheir collaborability or discussability. A text's studyabilityrelating to its richnessand depthmight also be important in instructional contexts. Wright (1980) hasmade a convincing argum ent that usability, which incorporates these critical contex-tual factors, w ould be a better criterion for designing and analyzing w ritten informa-tion. In many ways readability research emanates from the same instructional theorythat drives teacher-dominated lectures and recitations. The theory assumes that ifa teacher or a text is clear (or easy), the information will be transferred. If conceptsare difficult, the teacher (or text) only needs to repeat the information, perhapsusing shorter sentences or more familiar vocabulary. Research in the past twodecades has criticized this theory of human information processing, arguing thatlearning is a constructive, not a transfer, process (Bransford & Johnson, 1973;Britton, 1970; Bruner, 1986; Spiro, 1980).



6/28


TEXT STRUCTUREText structure research examines the relationship between text structure andcomprehension, usually measured by recall protocols (Crothers, 1972; Dawes,1966; Frederiksen, 1972; Kintsch & Keenan, 1972; Meyer, 1975, 1977, 1984).Researchers generally construct artificial texts, manipulating isolated text structurevariables and testing their effect on readers' comprehension by comparing thenumber of idea units in recall protocols to the idea units in the passage. Main ideasand superordinate information are said to be high in the text's content structure(depicted as a tree diagram ); details, elaborations, and subordinate information arelow in the text's structure. Structural signals such as headings, previews, logicalconnectives (e.g., first, second, the point is), pointer words (e.g., fortunately, moreimportantly), and summary statements provide important cues regarding the text'sstructure to the reade r. Textbook revisers might look to structure research to answerimportant questions regarding how texts are best organized for maximum recallabil-ity and how to best utilize structural signals, adjunct aids, and advance organizers.

Organizing a Text for Maximum RecallabilityNot surprisingly, studies reveal that as long as readers are aware of the text's

structure, a well organized text (one that follows rhetorical schemata such as cause/effect or comparison/contrast) is better remembered than a disorganized or ran-domly organized text (Kintsch & Yarbrough, 1982; Meyer & Freedle, 1984; Rich-gels, McGee, Lomax, & Sheard, 1987; Taylor & Samuels, 1983). In addition,readers w ho organized their recalls according to the tex t's structure remember m orethan those who do not (Meyer, Brandt, & Bluth, 1980; Taylor, 1980).Studies have also found that the rhetorical form itself can affect recall. Meyerand Freedle (1984) found that college students better recalled those texts which

followed the more organized forms of comparison and causation than those textswhich were organized as a collection of descriptions. However, Richgels et al.(1987) found that sixth-grade students were less able to use the causation structureas a recall aid than collection, comparison/contrast, and problem/solution.In addition, most studies have found that information high in the text's struc-ture is more likely to be recalled than information low in the text's structure(Britton, Meyer, Hodge, & Glynn, 1980; Britton, Meyer, Simpson, Holdredge, &Curry, 1979; Duffy, Higgins, Mehlenbacher, Cochran, Wallace, Hill, Haugen,McCaffrey, Burnett, Sloane, & Smith, 1989; Kintsch & Yarbrough, 1982; Meyer,1975, 1977; Meyer et al., 1980; Richgels et al., 1987). High-level information isalso more likely to be retained over time, for example, when the delay in the freerecall protocol is as long as one week (Meyer 1975, 1977). Meyer also found thatthe patterns of logical relations in top-level content were powerful in predictingwhich top-level units would be recalled, whereas the pattern of relations low in



7/28


content structure had no influence over which low-level units were recalled (M eyer,1975; 1977).The question as to why high-structure information is better recalled is morecomplex. This question is important because some research findings call into ques-tion the necessity of low-level structure in textbooks at all. Hidi and Baird (1988)found that salient elaborations did not improve the learning of a text's more generalmain points. Reder and Anderson (1980) found that reading summaries alone re-sulted in posttest performance as high as that obtained when entire chapters wereread. Other studies, however, have found that the presence of details enhanced therecall of main ideas (Phifer, McNickle, Ronning, & Glover, 1983). Britton et al.(1980) found that low-content information was remembered as well as high-contentinformation (even though it was missing from recall protocols); it simply was notas easily retrieved from memory. Retrieval cues helped make the low-contentinformation more accessible.

The ability to recognize text structure and use it as a recall strategy increaseswith age and reading ability (Englert & Hiebert, 1984; McGee, 1982; Meyer etal. , 1980; Richgels et al., 1987; Taylor, 1980, 1982; Vipond, 1980; Williams,Taylor, & de Cani, 1984). Younger readers may not recognize structural cues andsignals; Taylor (1982) found that fifth-graders had trouble mastering a readingstrategy focusing on text structure. Williams, Taylor, and de Cani's study (1984)revealed a clear developmental trend in recognizing and responding to text macro-structure or gist in their study of third, fifth, and seventh graders and adults. Inother studies, low ability readers were less aware of text structure or less likely toutilize structural cues to aid recall than were high ability readers (Meyer et al.,1980; Vipond , 1980). Studies of college undergraduate readers showed that theywere adept in a structural reading strategy (Britton et al., 1989; Meyer, 1975 , 1977;Vipond, 1980). Meyer (1975, 1977) found that, regardless of ability, high-structureinformation was better recalled than low-structure information.The effectiveness of a text's structure on recall is sometimes enhanced by textstructure instruction (Berkowitz, 1986; Meyer et al., 1980; Slater, Graves, & Pich,1985; Tay lor, 1980, 1982). Although students can be taught this skill, its usefulnessin improving recall seems to be passage (or passage difficulty) dependent (Taylor& Beach, 1984). Structure instruction is most effective when students are requiredto actively reconstruct a text's structure through activities such as making conceptmaps or filling in outline grids (Berkowitz, 1986; Slater et al., 1985). Other typesof instruction which encourage students to actively reconstruct the text, such asdiscussing a tex t, can be as effective as structure instruction, especially on com pre-hension measures other than recall (Slater, Graves, Scott, & Redd-Boyd, 1988).

Yet text structure is not the only factor affecting which information is remem-bered. In a study contrasting passages with the same structure but different con tent,Meyer (1977) found that, with college undergraduate readers, text structure only



8/28


partially predicted (r = .55 , r-squ are d= .30) the probability of units in the passagebeing recalled. Meyer acknowledged that "the nature of the content . . . appearsto have a substantial effect on whether these items will be well or poorly recalled' '(1977, p. 324). Meyer then reported that numbers were particularly apt to berecalled.

How a reader reacts to the content can overpow er text structure effects. In onestudy, Meyer and Freedle (1984) found that recall of high-level information wasunusually low when subjects (teachers) strongly disagreed with a passage whichargued that a school board immediately dismiss coaches who require athletes tolose body water weight. Other researchers, though not specifically examining textstructure effects, have evidenced that readers' expectations can override a text'sunderlying logic. Spiro (1975), for example, found that recall errors could beinduced when subjects were presented with readings that contradicted their expecta-tions. Similarly, Alvermann, Smith, and Readence (1985) found that when sixthgraders read counterintuitive science texts on heat and light, their prior misconcep-tions generally overrode the conflicting (correct) text information.

The difficulty of a text also can limit the effects of text structure. In a reviewof the literature, Roller (1990) presented evidence for an interaction effect betweenprior knowledge and structure variables. Studies showed that text structure had aneffect when a text was moderately unfamiliar, but not when a text was either verydifficult or was on a topic with which readers were familiar. Roller argued that ifthe text is on a moderately unfamiliar topic, the reader can use the structure toconstruct the relations between the tex t's concep ts. If the text is on a familiar to pic,the reader already knows the relations betw een concepts. If the text is difficult, itsstructure is not enough help for the reader to be able to construct the relationshipsbetween concepts.Several studies have tested the comprehension models of Kintsch and van Dijk(1978) and Kintsch and Vipond (1979) which describe comprehension as involv-ing dual micro- and macrolevels. Lexical and inter-sentence cohesion would aidmicroprocessing; rhetorical predicates (statements that specify the structuralrelations among the ideas) would aid macroprocessing. Implications for text revi-sion would then be that cohesive and coherent texts would aid micro- and macro-processing.Roen and Pich (1984), however, determined that various combinations of textforming (micro and macro) structure had unpredictable effects on comprehension.Neither microvariables nor macrovariables had any systematic effect on recall orreading rates. They wrote (quoting de Beaugrande) that perhaps "what is moreimportant in the formation of a coherent discourse is not the explicitness of micro-level textual information but the 'underlying connectivity of text-knowledge andworld-knowledge' " (p. 20). Regarding macrovariables they wrote that "it maybe that the effects of rhetorical predicates are so context bound that they cannot be



9/28


predicted across different text types or topics" (p. 20). Thus, their work pointsagain to the importance of the context (such as readers' prior knowledge andexpectations and text content) in determining the effectiveness of instructional textrevisions.Structural Signals

Interestingly, studies focusing on the effects of structural signals have notrevealed clear relationships between highlighting the underlying logical structureof a text and improving recall (Britton, Glynn, Meyer, & Penland, 1982; Meyer,1975). Meyer (1975), for example, found signals (structural cues, previews, sum-mary statements, and pointer words) to be of no significant aid to college readersin recalling text. Yet there is evidence that these implicit signals aid readers' deeperunderstanding. Britton et al. (1982), studying college students, found no recalleffects using M eyer's (1975) same signals and tex ts, but they did find faster reactiontimes with the signaled text in a secondary task (responding to a clicking sound byreleasing a telegraph key). Loman and Mayer (1983) claimed that recall was notan adequate measure of signal effect. Signals, they argued, aided readers' under-standing of conceptual and implicit relationships, understandings that recall testsdid not measure. In their study of lOth-grade good and poor comprehenders, theyfound significant effects for signaled passages (combining previews, headings,and logical connectives) with both groups in a problem-solving test. Recall andrecognition tests found no significant differences between the signaled and unsig-naled passages.

In an attempt to untangle and clarify signal effects on relatively sophisticatedcollege readers, Spyridakis and Standal (1987) designed a study which: (a) isolatedsignal types (headings, previews, and logical connectives), (b) guarded againstaltering the syntactic complexity of the original or control passage, and (c) testedsignal types' main effects and interactions on eight versions of four technical pas-sages of differing length and difficulty. Comprehension was measured by amultiple-choice test measuring understanding of not only detail and superordinateinformation but also implicit relationships. They found that while none of thesignals (headings, previews, and logical connectives) inhibited comprehension inthe different passages , neither did they predictably increase it. AH results wereinstead passage dependent, with previews and logical connectives having significanteffects in two of the four passages. They concluded that the likelihood of demon-strating strong and consistent results increases when one uses passages of somelength and difficulty on unfamiliar topics (which further confirms Roller's [1990]argument that there is an interaction between prior knowledge and structure vari-ables). Passage detail (recall) and inference scores also varied, which led Spyridakisand Standal to agree with Loman and Mayer's (1983) contention that recall is apoor measure of signal effects."Qualitative questions," they wrote, "elicit the



10/28

RevisingInstructional Text 315

effects of signals in a way that counting idea units alone in recall situations maynot" (Spyridakis & Standal, 1987, p. 293).Adjunct AidsAdjunct aids (learning objectives, preview s, summ aries, and questions em bed-ded in a text or inserted before or after a passage) are a common characteristic ofinstructional text. Studies investigating their effects have revealed the limitationsof providing readers with explicit directions for understanding and interpreting atext. In one study, researchers found that learning objectives resulted in no signifi-cant gain (as measured by a 25-item posttest) in either intentional or incidentallearning (Glover, Zimmer, Ronning, & Petersen, 1980). In a study of the effectsof a total package of structural aids (learning objectives, previews, summaries, andquestions) as is typically found in school textbooks, Zimmer (1985) found noimprovement in readers' performance on immediate and delayed multiple-choiceand cued recall tests. He hypothesized that these aids might actually interfere withcompetent readers' natural strategies and suggested "that a considerable portion ofthe variance in adjunct aid research might be attributable to the 'match' betweenthe strategies imposed and the measures for assessment" (p. 232).

This interference hypothesis is confirmed in Anderson and Biddle's (1975)review of research on adjunct questions. They found that adjunct questions weremore effective when they appeared after the relevant passage than when they ap-peared before it (p. 94). In addition, questions inserted after a section of textconsistently improved performance only on those posttest items which repeated thequestions at a verbatim level and not on those posttest items which paraphrasedthe adjunct questions. This finding, Anderson and Biddle (1975) wrote, "leavesopen the possibility that the effect is trivially specific" (p. 126); they concludedthat "some, but not all of the direct questioning effect depends upon reproducingthe specific language of the adjunct questions" (p. 127).Asking higher order (such as inference, analysis, and evaluation) postquestionsgenerally facilitates recall performance yet the conditions under which this facilita-tion occurs are not well understood (Rickards & Di Vesta, 1974; see Andre, 1979,and Rickards, 1979, for reviews of this research). Adjunct aids can have the effectof focusing a reader's attention on some parts of a passage to the exclusion ofothers (Andre, 1979; Dee-Lucas & Di Vesta, 1980; Melton, 1978; Sagaria & DiVesta, 1978); higher order questions may facilitate recall by directing readers'attention to more information (Andre, 1979). Andre noted that directing readers'attention through questions and other adjunct aids

will have a maximum effect when the subject perceives his task to be one ofgetting through the instructional materials with a minimum of effort . . . . If,however, the subject perceives his task to be one of learning the maximum amountfrom the material, questions and other adjunct aids may not facilitate his perfor-mance, (p. 288)



11/28


Apparently, predicting adjunct aid effects is difficult; the right type, amount,placement, and combination continues to be influenced by readers' purpose, howtheir performance is measured, and passage difficulty. Interestingly, whereas sig-nals were shown to aid performance on problem-solving tests (Loman & Mayer,1983), adjunct aids may have a detrimental effect on such problem-solving mea-sures (Zimmer, 1985, p. 230). This paradox could perhaps be explained by as-serting that the purposes and functions of signals and adjunct aids are quite differ-ent. Signals serve to clarify the underlying logic of a text and, thus, while theymay not improve recall of low-level information, they may promote better under-standings of underlying concepts and implicit relationships. These understandingscould then be seen to benefit students who are asked to read moderately difficulttexts and then solve problems. The purpose of adjunct aids, however, is to insurethat students arrive at a standard interpretation of the text. By interpreting the textcorrectly, students would then score well on end-of-chapter and unit tests, but theywould be at a disadvantage when asked to make sense of the text and use it forthemselves.

Advance OrganizersResearch on the effects of advance organizers in texts has not been conc lusive.Ausubel (1968) defined advance organizers as "approp riately relevant and inclusiveintroductory materials . . . introduced in advance of learning . . . and presented ata higher level of abstraction" (p. 148). Barnes and Clawson (1975) reviewed 20out of 32 studies and concluded that "advan ce organizers . . . do not facilitatelearning," but this conclusion has been criticized by Lawton and Wanska (1977)and Mayer (1979). Lawton and Wanska (1977) pointed out that, theoretically,advance organizers are only potentially effective, their effectiveness being depen-

dent on learners' prior knowledge and concepts. Advance organizers, Mayer ar-gued, are effective only when learners do not have or do not use prior knowledgesubsumers: "The best test of advance organizers occurs when material is unfamil-iar, technical or otherwise difficult for the learner to relate to his or her existingkn ow ledg e" (Mayer, 1979, p. 37 2). Mayer also pointed out that the studies Barnesand Clawson (1975) reviewed used recall or achievement tests to measure what islearned, yet advance organizer theory predicts that advance organizers will facilitateconceptual retention and far transfer at the expense of retention of specific technicaldetails and near transfer (Mayer, 1979).Mayer's (1979) own research found advance organizers effective in facilitat-ing conceptual retention of unfamiliar technical material. In one study, collegefreshmen who had no prior computer programming experience read a 10-page texton a simple programming language. One group read a text that began with a modelof the computer and explained each program statement according to the model; theother group read the same facts without the model. Results indicated that the



12/28


advance organizer led to better performance on far transfer measures (such asinterpreting an 8-line program) but poor performance on near transfer measures(such as writing a single statement to solve a given task).In short, advance organizers may be potentially effective in facilitating concep-tual retention, depending on readers' prior knowledge and perhaps both the perfor-mance measures and the nature of the material to be learned. Mayer (1979), forexam ple, suggested advance organizers are best suited for technical materials, suchas science and math texts. In many ways the effects of advance organizers echothe effects of text structure discussed earlier. Both have a history of confusing andcontradictory findings, but when one considers the interaction effects of readers'prior knowledge, a clearer picture emerges. This similarity should come as no

surprise since both advance organizers and text structure seek to make conceptualrelationships more explicit.Text Structure Research and Instructional Text Revision: Conclusions

Although the research has thus revealed the powerful potential influence textstructure can have as an aid to memory, one might want to keep in mind findingsthat indicate the influence of variables that lie outside of the text. One such set ofvariables lies in the reader. The interaction between readers' prior knowledge andstructure variables has been d iscussed. A lso discussed was the reader's affectiveresponse. If information in the text is against readers' beliefs or contradicts theirexpectations, then recall is lower, for readers w ill let their own personal constructsoverride the author's organizational pattern (Alvermann et al., 1985; Meyer &Freedle, 1984;.Spiro, 1975). Additional reader variables influencing recall are age,interests, goals, and culturally-based expectations (Andre, 1979; Holland, 1981).

Another set of overriding variables involves the extent of readers' generativeactivities or, simply, how actively a reader reads. For example, the simple act ofgenerating a main idea sentence for a non-topicalized passage boosts recall to thatof topicalized passages (Bridge, Belmore, Moskow, Cohen, & Matthews, 1984).Research also shows that when readers generated topic sentences, headings, andrelated and unrelated sentences, performance on free recall, passage knowledge,and passage structure tests increased (Dee-Lucas & Di Vesta 1980). Readers whoconstructed concep t m aps (Berkow itz, 1986) or filled in outline grids (Slater et al.,1985) also performed better on recall mesures than groups who simply studied themaps or outlines.Although research indicates that a reading strategy which focuses on text struc-ture may improve recall, the strategy may have its limits. One major limit is thelow ecological validity of the short, experimentally contrived texts. Students arerarely taught major concepts by reading just one short textbook passage in a singlesitting. M ore likely , they are asked to read entire chapters over the course of a fewdays or even weeks. Furthermore, the experimental texts, tightly constructed by



13/28


structure researchers to match strict hierarchies, are the exception, not the rule inexpository texts students will encounter in magazine or newspaper writing and inmany content area textbooks. This may provide an argument to revise textbooksaccording to a strict structural hierarchy; however, this might then handicap stu-dents when it comes to finding an appropriate strategy for comprehending real-world writing. Hare, Rabinowitz, and Schieble (1989) posited this argument afterthey found that sixth- and eleventh-grade students were able to infer significantlyfewer correct main ideas for less constrained (content area textbooks) texts thanfor the contrived (basal skills lessons) texts. That Hare et al. (1989) consideredcontent area textbooks' structure as less constrained is relative; Ashton, O'Hear,and Pherson (1985) and O'Hear, Ramsey, and Pherson (1987) were able to showthat college sociology and composition textbooks made structure explicit.Braddock's (1974) study of the frequency and placement of topic sentences innaturally occurring prose, such as essays in The Atlantic, Harper's, and The NewYorker, revealed that the structure of this expository writing tends to be implicit;hierarchical paragraph ordering led by a simple topic sentence was evident in only13% of contemporary writers' paragraphs. The process of identifying the topicsentence (and thus to some extent determining the text 's structure) w as difficult forBraddock, an English academic. One wonders what strategies readers then use tocomprehend such texts and whether attempting to use a structure strategy in suchcases would be recommended. Braddock (1974) and Hare et al.'s (1989) findingscall into question the wisdom of improving instructional texts by making theirstructure more explicit. Perhaps texts' structure should be less explicit so as toencourage the developm ent of a more flexible repertoire of reading strategies (Hareet al., 1989).

A second limitation of much of the text structure research is its dependencyon recall as a measure of comprehension. Many would argue against this equatingcomprehension with remembering, specifically, with recalling text information.Although memory may play an integral role in comprehension, it is not an equiva-lent process (Carroll, 1972). Comprehension, like memory, is an inferential recon-structive process w hich is affected by a variety of impinging contexts (Spiro , 1980).One's understanding of a text is not likely to match the text itself. Thus, the aboveresearch is limited, for although it tells us the amount of match between students'memory and the text information, it says nothing about what students comprehend,or what text characteristics might help students critically think about the material.If the dependent measure were based on a request to debate the text informationor to write an analysis of it, one wonders if a contrived text structure would besuch a powerful predictor of high performance.

These studies, then, do not take into account the importance of readers' goalsand purposes in both shaping their reading strategy and determining the effective-ness of a text. For example, under recall conditions, a reader's purpose is toremember as much of the information as possibleunder that condition mostreaders will realize the advantage of using the writer's organization as a recall aid.by guest on May 1, 2013jlr.sagepub.comDownloaded from


14/28

Revising Instructional Text 3 ] 9

One must ask: Is success in ins tructional contexts based mostly on one 's abil i ty toreca l l tex ts? Al though in many c lass rooms the answer i s yes , in many o ther c lass -roo m s the em ph asis is on crit ica l thinking , which is quali ta t ively different fromrecall; that is , i t is not simply the next step after recall. (Though end-of-chapterquestions , with recall quest ions f irs t and inferentia l quest ions fol lowing, wouldlead one to believe otherwise .) Given that nationwide tes ts (Applebee, Langer, &M ull is , 1989; Na t iona l Assessm ent for Educa t iona l Prog ress , 1981) ev iden ce tha ts tudents succeed in recall ing basic facts but do poorly when asked to cri t ica l lythink about these facts , textbooks ' recallabil i ty might not be such a high priori tyfor educators. An explicit text structure may, in fact, facilitate recall while at thesame t ime hinder s tudents ' abil i ty to make sense of the information according tothe i r own cogni t ive s t ruc tures . Al though making the i r own sense of the tex t mayresu l t in some misunders tandings , i t i s those very misunders tandings which canthen be addressed in the c lassroom or in the text itself. A contrived recallable textmay provide l i t t le opportunity for s tudents to see how the information can beincorporated into their own cognit ive s tructures . Perhaps this explains the welldocumented problem (Carey , 1986) of pervas ive under ly ing misconcept ions he ldby science s tudents after they have successfully passed re levant sc ience courses .

Other l imita t ions of this research are s imilar to the l imita t ions of readabil i tyresearch: Too much has been left out of the equation. Good ins tructional texts aremore than readable and more than recallable . A good ins tructional text should bestudyable; that is , i t should s t imulate interes t in and cri t ica l examination of i tssubject . I t should a lso be enjoyable; would s tudents ever voluntari ly e lect to readthese perfectly s tructured texts? A good ins tructional text should a lso be able tomake diff icult concepts unders tandable , yet we know li t t le about how studentsmake sense of a text 's explanation of diff icult and complex concepts . Thus, a f inall imita t ion of these s tudies is that their decontextualized f indings have l imited valid-i ty in c lassroom sett ings when one takes into account the importance of context indef in ing and shaping read ing , w ri t ing , and th ink ing processes (Barr , 1987 ; B loo m e,1987; Blo om e & G reen , 1982; Herr in g ton , 1985; M cC arthy , 1987; Ne lson , 1990) .The socia l contexts of teaching and learning are cri t ica l . We do not even know,for example , i f s tudents actually read their textbooks in the manner text s tructureresearchers assume ( f rom beginning to end) , fo r many c lass room contex ts encour-age s tudents to skim texts in order to effic iently locate answers to end-of-chapterquestions . Future research in improving ins tructional text wil l thus have to examinetextbooks in c lassroom contexts , focusing on reader-text-context interactions inorder to better unders tand the powerful forces shaping how students comprehendinstructional text and how we might go about improving ins tructional writ ing.

T E X T I N T E R E S T I N G N E S SA small body of research has focused on how the interes t ingness of texts

affects comprehension. Researchers point to the known effect of interes t on memoryby guest on May 1, 2013jlr.sagepub.comDownloaded from


15/28


and hypothesize that if a text was interesting, students would find it easier torecall (Anderson, Shirey, Wilson, & Fielding, 1986; Hidi & Baird, 1986, 1988).Interestingness may also explain some of the passage dependent findings reportedabove in text structure research. Supporting the effects of idea saliency are NationalAssessment of Educational Progress (NAEP) analyses which found that Black andHispanic students scored significantly higher than their White peers on questionsconcerning Black and Hispanic history and literature (National Assessment of Edu-cational Progress, 1987). Of course, this finding also points to the relativeness ofsaliency. Luftig and Greeson (1983) also found some evidence for high saliencyidea units being better remem bered than low or medium saliency units. This findingis hard to interpret, however, since saliency was determined by raters other thanthe subjects.Anderson et al. (1986) have suggested four attributes that may contribute tosentence interest:1. Character identification: People are more interested in characters with whomthey readily identify.2. Novelty: Novel or unusual content will enhance interest.3. Life theme: People are interested in what is important to them.4. Activity level: Material that describes intense actions and feelings is more inter-esting than static scenes and less intense states.

Using A nderson et a l. 's (1986) guidelines and two other interest-evoking strate-gies, Hidi and Baird (1988) tested how interestingness affected fourth and sixthgraders' recall of a history textbook passage. They followed three interest-evokingstrategies to create three texts. The first strategy was to incorporate three of thefour attributes (life theme could not be manipulated) to make a base text. In thistext, for example, they elicited character identification by including facts such as"Thomas Edison became the most famous inventor of all time even though he leftschool when he was very young."

Their second strategy was to insert salient descriptive elaborations after severalof the important ideas in the passages. Thus, after stating that an inventor has tohave a strong interest in what he is doing, they added, "Sometimes they becomeso interested they would forget to go home to eat dinner. ' ' They added this salientelaboration to the base text to create a salient text.Their third strategy was to manipulate the text so as to induce a need on thereader's part to resolve some incomplete understanding of new information. Thus,after stating how inventors before Edison had failed to make a light bulb thatactually worked, they wrote,"How did Edison make a better light bulb?" Theresolution that answered this question was then given after an intervening para-graph. This resolution text was created by modifying the salient text.Presumably these three strategies could serve as guidelines for instructionalwriting revision. Results were that the combined recall for the three passages was



16/28


significantly higher than for a standard text. There was also significantly less forget-ting (measured by a 1-week-delayed recall protocol) for the three passages. Thepassages using the second and third strategies did not result in any significantincrease over the interesting base text, though the students' ratings indicated anincreased subjective rating for these two passages. A final (unsurprising) findingwas that the interest-evoking strategies were most effective in increasing children'srecall of concrete, specific, or personally moving information, but did not enhancethe acquisition of the less interesting and more abstract, general, or scientific infor-mation.

Certainly making text interesting has value even if it does not result in higherperformance on recall tests since interesting texts are more likely to be read andenjoyed by students. One limitation is, of course, the relative nature of interest-ingness. Here, Anderson et al. (1986) and Hidi and Baird (1986, 1988) have madegood progress in defining and operationalizing interestingness. A second limitationis that the above strategies of interestingness seem more appropriate for improvingsome kinds of instructional texts, such as social studies texts, but less so for moretechnical material such as chemistry, physics, and mathematics texts. Creatinginterestingness in these other texts could conceivably use some of the same strate-gies but would also require new strategies, a task for further research. A relatedissue is that instructional texts often must be more than just interesting. In manycases they must also be able to effectively explain difficult concepts. This require-ment may be more evident in a physics text than a social studies text. In the caseof a physics text, interestingness may be a necessary but not sufficient conditionfor a text's effectiveness.

A final concern, which was also noted in the Text Structure section, is thedependence of this research on recall protocols to measure effects of interest-ingness. If memory and comprehension are inferential reconstructive processes asSpiro (1980) theorized, the match between readers' recalls and text informationwould be an inappropriate way of m easuring the effects of interestingness. A bettermeasure of comprehension needs to be found, one that can show how students areactually responding to and making sense of the passage information.

EXPERT REVISERS' STRATEGIESThis fourth section will focus on research that examines the effects of expertrevisers' strategies. The notion behind this research methodology is that effectiverevision guidelines and general principles could be developed by studying experts'revisions and their effects on comprehension.The strategies of expert revisers have been examined by giving revisers apassage and telling them to revise it in such a way as to improve performance onsuch measures as recall protocols, recognition, and factual and inference tests. A



17/28


main finding has been that expert revisers often improved the retention of texts butthat these improvements were not consistent across all the texts they revised; also,what revisers said they did to improve texts often did not correspond to how thetexts were actually improved (Britton et al., 1989; Duffy et al., 1989). In manycases, revisions were not improvements at all (Britton et al., 1989; Duffy, Curran,& Del Sass, 1983; Duffy et al., 1989; Graves, Slater, Roen, Redd-Boyd, Duin,Furniss, & Hazeltine, 1988; McLean, 1986). Britton et al. (1989) found that inrevisions of 10 army training manual passages, only three of revisers' eight strate-gies actually corresponded to improved texts. The three strategies all related tosignaling structure and together accounted for 50% of the variance in performance;the dependent variable was a 24-hour delayed recall protocol.

When one looks at revisers' changes and their actual effects, results are incon-clusive. Revisions that in essence simplified a text (determined by readabilityscores) were not effective in Britton et al.'s (1989) first study, but were effectivein their third study. In their second study, the more complex texts were actuallybetter retained. Some revisions resulted in higher performance than other revisionsin recall but not in recognition tests whereas other revisions resulted in higherperformance only in delayed recalls and lower performance for the immediate recall(Britton et al., 1989).Research in revisers' strategies repeatedly hint that the reader's purpose influ-ences whether or not a revision is successful. Since retention was what was mea-sured, it is not surprising that the strategies which highlighted text structure wereusually successful. Interestingly, in one study students rated the success of revisionsdifferently depending on if they took a fact or an inference test (Duffy et al.,1983). Other probable factors influencing the success of a revision were readers'expectations and passage difficulty (Britton et al., 1989; Duffy et al., 1983).Aside from finding that, even for expert revisers, improving texts is a hit and

miss operation, these studies focusing on revisers' strategies do not provide us withany new information about how to improve instructional writing. The findingsmerely corroborate the im portance of text structure; yet this should not be a surprisesince the readers' purpose in most studies was to recall the passage content. Thesestudies are still useful, how ever, for they indicate a need for systematic and theoreti-cally based research in developing effective revision strategies for instructionaltext.

READERS' COMPREHENSION STRATEGIESStudies looking at readers' strategies using verbal protocols as they try tounderstand difficult text have produced more fruitful findings (Charrow, 1988;Charrow, Holland, Peck, & Shelton, 1980; Flower, Hayes, & Swarts, 1980; Hol-land & Redish, 1981 ; Sw aney, Janik, Bond , & Hayes, 1981; W right, 1980). Much



18/28


of this work was done by the Document Design Project at the American Institutesfor Research, Carnegie Mellon University, and Siegel & Gale, Inc., for the purposeof making public documents easier to understand and use.Charrow et al. (1980) observed problems Medicaid recipients had in under-standing and using the Medicaid Recertification Form. The recipients, who weretypically poor, old, and not well educated and who often spoke a non standarddialect or were not native speakers of English, often m isinterpreted the term "fam -ily members." The form writers intended the term to refer to immediate, nuclearfamily members, but recipients often interpreted it to mean extended family mem-bers. By studying these readers' understandings, the form was revised to be morecompatible with their understandings. In another study aimed at revising functional

documents (e.g., regulations, contracts, policies) Flower et al. (1980) found thatreaders radically restructure text information, not merely try to rephrase a statementin a simpler way. Their restructuring involved building scenarios, that is, theyreorganized the information around human agents and their actions; they also at-tempted to instantiate those scenarios, that is, to provide the context and concreteinstances that make a general rule more understandable. This scenario strategy wasalso corroborated by protocols in Swaney et al.'s (1981) work in revising carinsurance policies. Instantiation is not merely providing examples but involvesa process that is broader than simply giving an example. . . . [It] suggests the actof filling in the slots in an abstract framework in much the way people use familiarmental scripts or schmas to make sense of new information. Such schmas areonly empty structures that become meaningful when they are "instantiated" witha specific, concrete reality. (Flower et al., 1980, p.30)Some examples of these instantiated scenarios are:Document: No part of such financing shall be used. . . .Scenario: If a borrower has that money on hand, . . . we won't make her a loanso that she can use that money on hand to purchase land or things like that.Document: (Discussion of eligible concerns) Printing: A firm solely engaged incommercial or job printing, if there is not common ownership with a concernineligible under this paragraph (d) (4) of this section and the printer had no directfinancial interest in the commercial success of the material so produced.Scenario: You have something that you need printed, you bring it in, they printit up. (Flower et al., 1980, p. 16)Medicaid caseworkers have also been observed clarifying Medicaid Forms forrecipients by making scenario-like translations. For example, for the form item,"amount of support from friends or relatives" caseworkers would say, "Does afriend or relative give you money, food, or a place to live?" (Holland, 1981).Flower et al. (1980) have devised revision guidelines based on these readers'



19/28


strategies, organizing information around agents and actions placed in a context(the scenario principle). They noted that the primary goal of old regulations wasto teach a specified body of information, and that "like traditional expositorywriting of the kind found in textboo ks, it was rigorously o rganized around the topicat hand" (p. 22). The revisions involved itemizing, turning labels into questions,and providing scenario-like translations. They found that a panel of judges couldeasily distinguish revised from old headings:

Old: Personal Liability; Occurrence; Social DisadvantageRevised: Who is an eligible student borrower? The Loan Application Process; CanThis Policy Be Canceled? (Flower et al., 1980, p. 28)The guidelines ask that information be instantiated because this was the wayreaders made sense of difficult concepts. Thus, the insurance policy concept of"Broader Coverage" is instantiated by this illustration:You've boarded your neighbors' poodle while they're away on vacation. You'recareless and the poodle runs away and gets lost. Your neighbors insist on yourpaying for the loss. If he was an ordinary poodle worth say $400, you pay $200and we pay $200. But if he was a prize-winning show dog worth $4,000, you pay$250 and we pay $3,750. (Flower et al., 1980, p. 30)

Less extensive instantiation is also possible as in this example (the instantiation isitalicized):In addition, certain non-students (such as doctors serving as interns or resi-dents). . . . (Flower et al., 1980, p. 31)What is interesting about these examples is that they restructure informationin accordance with how the experienced readers made sense of the text. Wonderinghow far this scenario principle could be generalized, Flower et al. (1980) had

subjects read expository prose from a textbook and found that 40%-50% of theirthink-aloud clauses were scenarios, indicating that experienced readers of instruc-tional text relied on the same sense making strategies as skilled readers of functionaldocuments. They concluded: "Perhaps scenarios are simply a way people translateany sort of difficult prose in an attempt to make it comprehensible" (p. 34).What should be noted with Flower et al.'s (1980) work is their use of verbalprotocols to provide a richer and more complete examination of comprehension.In their study, the readers' purpose was to understand the text, not simply recall

it. This purpose is presumably more in line with why people actually read , althoughone could argue that in many school contexts students' purpose is to recall informa-tion. The revisions themselves are successful. Swaney et al. (1981) reported thateven though the protocol-aided revisions were up to four times longer than theoriginals, they were read more quickly, did not receive the original's negativecomments, and comprehension error rates decreased from 50% to 22%.Unfortunately, Flower et al. (1980) did not test how the scenario principles



20/28


might best be applied to revising instructional writing. Yet Flower has continuedto use think-alouds as a way to see how readers understand text. A recent study(Haas & Flower, 1988) found that only experienced (graduate student) readersfollowed a rhetorical reading strategy in which text information was restructuredand instantiated. Less experienced readers (college undergraduates) were morelikely to merely rephrase the text's content or note a text's structure, strategieswhich would most likely help them simply recall information. Haas and Flowerargued that a rhetorical reading strategy is necessary if students are to understanddifficult and complex texts as well as be critical readers.

Brown and Clement (1987) also used difficult texts and reader think-alouds intheir study, although they used the think-alouds to test the effectiveness of analready revised text rather than using the think-alouds to guide the revision. Theystudied how high school students understood an innovative physics textbook'sexplanation of a commonly misunderstood concept (Newton's Third Law whichstates that to every action there is always opposed an equal reaction). They com-pared this textbook's explanation which used a number of examples to illustratethis law with an experimental explanation which used bridging analogies as expla-nations. Their hypothesis was that the commonly used illustrative examples in thetextbook would be less effective than the bridging analogies in teaching studentsthe concept.The dependent measures were: (a) questions which aimed to measure bothstudents' conceptual knowledge and their confidence in their knowledge, and (b)interrupted think-alouds in which readers were periodically asked how much sensea particular example or bridging analogy made.Results were favorable for the experimental explanation, showing that howone instantiates a difficult concept makes a difference in learning. The data fromthe interrupted think-alouds in their study supported three guidelines for writinginstantiations:

First, the anchoring examples used must make sense to the students, not simplyto the teacher or textbook author presenting them. Second, analogical relationshipswhich are obvious to the instructor [or author] need to be explicitly developed forthe student. Third, it may be important to develop qualitative models which givemechanical explanations for phenomena [such as how a static object exerts force],(p. 14)These studies which use verbal protocols of difficult texts to examine howreaders understand difficult concepts have provided interesting information abouthow students' comprehension strategies interact with the text. This information ispotentially useful in developing instructional text revision principles. Further re-search should examine the effects of Flower et al.'s (1980) scenario principle andBrown and Clement's (1987) guidelines in making complicated concepts moreunderstandable for a variety of readers of different ages.



21/28


CONCLUSIONSIt may be that there is no ideal instructional text independent of variables such

as the setting in which the text will be used and the purposes, attitudes, priorknowledge, interests, and abilities of its readers. Each research tradition in thisarea has concentrated on different aspects: (a) readability research has focused onmaking sure the text is appropriate to the reader's age and abilities, (b) structuralresearch has focused on making sure the text is logical and clear, (c) interestingnessresearch has focused on making sure the text attracts readers' interest, (d) researchon revisers' strategies has sought to characterize their expertise and measure itseffects on readers' comprehension, and (e) research on readers' strategies has fo-cused on making sure the text can explain difficult concepts in a way readersunderstand. Of course, all this research works towards better characterizing andproducing good instructional text.

A limitation of much of this research is that little is known about how class-room contexts shape students' readings, about how these texts work in the field soto say. If it is true that the best representation of knowledge on a given topicdepends on how that knowledge will be used (Bobrow & Norman, 1975), then itmay be that the place to start in improving instructional texts is to examine texts'instructional contexts. But of course that alone would be insufficient. One wouldalso need to critically analyze the pedagogical theories underlying the writing ofthese texts.

What have we learned from this review? First, readability formulas, thoughproviding a simple and convenient quantitative index, are severely limited as revi-sion guidelines. T here are just too many critical variables that the formulas ignore .The formulas might best serve as screening devices for inappropriate texts, butsimply because a text is deemed readable (or unreadable) does not make it so. Inaddition, the practice of writing to formula is entirely ineffective. Thus, severalstudies have questioned the validity and adequacy of readability scores alone.Importantly, readability measures tell us nothing about a tex t's usability in effectiveinstructional contexts.

Text structure research has revealed that a tex t's organization can be a powerfulaid to memory. Thus, for students to best recall the ideas in an unfamiliar text, thetext should be hierarchically organized and logically structured. The main limita-tions of using text structure research to derive revision guidelines is that, in mostof the research, readers' tasks were limited to recalling the information and thetexts used were often experimentally contrived and not ecologically valid. Anotherlimitation is the decontextualized nature of this research. To improve instructionalwriting we must see how texts are used in effective instructional contexts. Finally,this research has evidenced a multitude of variables that overpower structure ef-fects. Thus, more research is needed to find out how readers' prior knowledge andculturally based expectations influence the way they understand a text as well ashow readers perform on tasks other than recall.



22/28


Research on making instructional text more interesting has had some successin defining revision principles that make text more interesting and memorable. Yetthis research has been limited by its use of recall m easures, w hich seem particularlyinappropriate measures of an interesting text's effectiveness. One must also keepin mind that interestingness is not an inherent or stable characteristic of a text;rather, it is dependent on a particular reader and a particular context.

Research on expert revisers' strategies has found that revisers' often do im-prove texts; however they do not consistently or predictably improve them. Thisresearch is also limited by its use of recall measures to determine texts' effec-tiveness.Some of the most com pelling research in this area has examined reader compre-hension strategies using verbal protocols to try to understand how readers compre-hend difficult text. This research has come up with successful revision strategiesfor improving text explanations of complex and difficult concepts by focusing onreaders' comprehension strategies and their conceptual understandings. PerhapsFlower et al.'s (1980) scenario principle will help develop a theory of discoursecomprehension that will guide text revisions to support an active reading strategyof conceptual understanding rather than the more passive strategy of text recall.In review ing th is research it has become more and more apparent that research-ers ' dependence on recall measures has limited their vision of what characterizesgood instructional text. Certainly recall is important in many instructional contexts,but it is not alone in its importance. Further research is needed to develop innovativemeasures of comprehension, such as think-aloud protocols (and interrupted think-alouds). One possibility is paraphrase methodology in which readers read a textand then explain their understandings onto audiotape (see Charrow, 1988). Anotherpossibility is pre- and post-reading interviews which would focus on how a reader'sknowledge was altered by the new information; yet, this measure would not giveresearchers the benefit of observing the immediate influence of sections of text onthe reader's understandings. Other ways to measure comprehension are suggestedin Schumacher and Nash's (1991) research on conceptualizing and measuringknowledge change. We may even need to alter our images and notions of whatcounts as comprehension. One might also wish to measure such qualities as atexts' discussability, studyability, or usability (see Schriver, 1986, for a review ofdifferent methods of user-testing).A final suggestion is that future research might focus on subject specific con-cerns. Researchers could begin by studying how different disciplines use texts anddetermining the typical areas of difficulty students have in these different disci-plines. It could be that social studies texts need to help students organize andrelate the people, places, dates, and events of history, while physics texts need tospecifically address common misconceptions.Finally, to emphasize an earlier point: Improved texts go hand in hand withimproved contexts. There is no successful revision principle apart from the setting

and purposes for which a text is read. Improving instructional writing thus demandsby guest on May 1, 2013jlr.sagepub.comDownloaded from


23/28


24/28


Bransford, J. D., & Johnson, M. K. (1973). Consideration of some problems of comprehension. In W.Chase (Ed.), Visual information processing. New York: Academic Press.Bridge, C. A., Belmore, S. M., Moskow, S. P., Cohen, S. S., & Matthews, P. D. (1984). Topicaliza-tion and memory for main ideas in prose. Journal of Reading Behavior, 16, 6 1 - 8 0 .Britton, J. (1970). Language and learning. Harmondsworth, England: Penguin.Britton, B. K., Glynn, S. M., Meyer, B. J., & Penland, M. J. (1982). Effects of text structure on useof cognitive capacity during reading. Journal of Educational Psychology, 74, 5 1 - 6 1 .Britton, B. K., Meyer, B. J., Hodge, M. H., & Glynn, S. M. (1980). Effects of the organization oftext on memory: Tests of retrieval and response criterion hypotheses. Journal of ExperimentalPsychology: Human Learning and Memory, 6, 620-629 .Britton, B. K., Meyer, B. J., Simpson, R., Holdredge, T. S., & Curry, C. (1979). Effects of theorganization of text on memory: Tests of two implications of a selective attention hypothesis.Journal of Experimental P sychology: Hum an Learning and Memory, 5, 496-506 .

Britton, B. K., Van Dusen, L., Gulgoz, S., & Glynn, S. M. (1989). Instructional texts rewritten byfive expert teams: Revisions and retention im provem ents. Journal of Educational Psychology, 81,226-239 .Brown, D. E., & Clement, J. (1987, April). Overcoming misconceptions in mechanics: A comparisonof two example-based teaching strategies. Paper presented at the annual meeting of the AmericanEducational Research Association, Washington, DC.Bruner, J. (1984). Language, mind, and reading. In H. Goelman, A. Oberg, & F. Smith (Eds.),Awakening to literacy (pp. 193-200). Portsmouth, NH: Heinemann.Bruner, J. (1986). Actual minds, possible worlds. Cambridge, MA: Harvard University Press.Carey, S. (1986). Cognitive science and science education. American Psychologist, 41, 1123-1130.Carroll, J. B. (1972). Defining language comprehension: Some speculations. In J. B. Carroll & R. O.Freedle (Eds.), Language comprehension and the acquisition of knowledge (pp. 1-29). Washing-ton, DC: V. H. Winston.

Charrow, V. (1979). Let the rewriter beware. Washington, DC: American Institutes for Research.Charrow, V. (1988). Readability vs. comprehensibility: A case study in improving a real document. InA. Davison & G. M. Green (Eds.), Linguistic complexity and text comprehension: Readabilityissues reconsidered. Hillsdale, NJ: Erlbaum.

Charrow, V. R., Holland, V. M., Peck, D. G., & Shelton, L. V. (1980). Revising a m edicaid recertifi-cation form: A case study in the docum ent design process. Washington, DC: American Institutesfor Research.Clifford, G. J. (1978). Words for schools: The applications in education of the vocabulary researchesof Edward L. Thorndike. In P. Suppes (Ed.), Impact of research on education: Som e case studies(pp. 107-198). Washington, DC: National Academy of Education.Coleman, M., & Liau, T. L. (1975). A computer readability formula designed for machine scoring.Applied Psychology, 60, 283-284.Crothers, E. J. (1972). Memory structure and the recall of discourse. In R. O. Freedle & J. B. Carroll(Eds.), Language comprehension and the acquisition of knowledge (pp. 247-284). New York:Wiley.Davidson, A., & Green, G. M. (Eds.). (1988). Linguistic complexity and text comprehension: Reada bil-ity issues reconsidered. Hillsdale, NJ: Erlbaum.Dawes, R. M. (1966). Memory and distortion of meaningful written material. British Journal of Psy-chology, 57, 77-86.Dee-Lucas, D., & Di Vesta, F. J. (1980). Learner-generated organizational aids: Effects on learningfrom text. Journal of Educational Psychology, 72, 304- 311 .Duffy, T. M. (1985). Readability formulas: What's the use? In T. M. Duffy & R. Waller (Eds.),Designing usuable texts (pp.113-143). Orlando, FL: Academic Press.Duffy, T. M., Curran, T. E., & Del Sass, D. (1983). Document design for technical job tasks: Anevaluation. Human Factors, 25(2), 143-160.



25/28


Duffy, T. M., Higgins, L., Mehlenbacher, B., Cochran, C., Wallace, D., Hill, C., Haugen, D.,McC affrey, M ., B urnett, R., Sloane, S ., & Sm ith, S. (1989). Models for the design of instructionaltext. Reading Research Quarterly, 24, 434-457 .

Duffy, T. M., & Kabance, P. (1982). Testing a readable writing approach to text revision. Journal ofEducational Psychology, 74, 7 3 3 - 7 4 8 .Elliott, D. L., & Woodward, A. (Eds.). (1990). Textbooks and schoo ling in the U nited States. Eighty-ninth yearbook of the National Society for the Study of Education (Part One). Chicago: Universityof Chicago.Englert, C. S., & H iebert, E. H . (198 4). Childre n's developing awareness of text structures in expositorymaterials. Journal of Educational Psychology, 76, 6 5 - 7 4 .

Flesch, R. (194 8). A new readability yardstick. Journal of Applied Psychology, 32, 221-233 .Flower, L., Hayes, J. R., & Swarts, H. (1980). Revising functional docum ents: The scenario principle(Document Design Project Tech. Rep. No. 10). Pittsburgh, PA: Carnegie-Mellon University.(ERIC Document Reproduction Service No. ED 192 345)Frederiksen, C. H. (1972). Effects of task-induced cognitive operations on comprehension and memoryprocesses. In R. O. Freedle & J. B. Carroll (Eds.), Language comprehension and the acquisitionof knowledge (pp. 211-246). New York: Wiley.Glover, J. A., Zimmer, I. W., Ronning, R. R., & Petersen, C. H. (1980). Nobody knows how toremember that prose. Journal of Educational Research, 73, 340-343 .Graves, M. F., Slater, W. H., Roen, D. R., Redd-Boyd, T., Duin, A. H., Furniss, D. F., & Hazeltine,P . (1988). Some characteristics of memorable expository writing: Effects of revisions by writerswith different backgrounds. Research in the Teaching of English, 22, 242-265 .

Haas, C., & Flower, L. (1988). Rhetorical reading strategies and the construction of meaning. CollegeComposition and Communication, 39, 167-183.Hare, V. C , Rabin owitz, M ., & Schieble, K. M . (1989). Text effects on main idea comprehen sion.Reading Research Quarterly, 24, 7 2 - 8 8 .Herrington, A . ( 198 5). W riting in academic settings: A study of the contexts for w riting in two collegechemical engineering courses. Research in the Teaching of English, 19, 331-359 .Hidi, S., & Baird, W. (1986). InterestingnessA neglected variable in discourse processing. CognitiveScience. 10, 179-194.Hidi, S., & Baird, W. (1988). Strategies for increasing text-based interest and students' recall ofexpository texts. Reading Research Quarterly, 23, 465-483 .

Holland, V. M. (1981). Psycholinguistic alternatives to readability formulas (Tech. Rep. No. 12).Washington, DC: American Institutes for Research.Holland, V . M ., & R edish, J. C. (1981 ). Strategies for understanding forms and other public docu me nts.In D. Tannen (Ed.), Proceedings of the Georgetown round table on discourse: Text and talk.Washington, DC: Georgetown University Press.International Reading Association. (1984/1985). IRA, NCTE take stand on readability formulae. Read-ing Today, 2(3), 1.Jackson, S. A. (1981). About publishers, teachers, and reading achievement. In J. Y. Cole & T. Sticht,(Eds.), The textbook in American society. A volume based on a conference at the Library ofCongress on May 2-3. 1979 (pp. 9-11). Washington, DC: Library of Congress.Johnson, L. L., & Otto, W. (1982). Effect of alterations in prose style on the readability of collegetexts. Journal of Educational Research, 75, 222-229.Kern, R. P. (1980). Usefulness of readability formulas for achieving army rea dability objectives:Research and state-of-the-art applied to the army's problem (Tech. Rep. No. 437). Alexandria,VA: U.S. Army.Kincaid, J. P., Fishburne, R. P., Rogers, R. L., & Chissom, B. S. (1975). Derivation of new readabilityformulas (Automated Readability Index, Fog Count, and Flesch Reading Ease Formula) for Navyenlisted personnel. Navy Training Com mand Research Branch Report, 8 - 7 5 .



26/28


Kintsch, W., & Keenan, J. (1972, April). Reading rate and retention as a function of propositions inthe base structure of sentences (Publication No. 6). Studies in mathematical learning theory andpsycholinguistics. Boulder, CO: Computer Laboratory for Instruction in Psychological Research.Kintsch, W., & van Dijk, T. A. (1978). Toward a model of text comprehension and production.Psychological Review, 85, 3 6 3 -3 9 4 .Kintsch, W., & Vipond, D. (1979). Reading comprehension and readability in educational practice andpsychological theory. In L. G. Nilsson (Ed.), Perspectives on memory research (pp. 329-365).Hillsdale, NJ: Erlbaum.Kintsch, W., & Yarbrough, J. C. (1982). Role of rhetorical structure in text comprehension. Journalof Educational Psychology, 74, 828-834 .Klare, G. R. (1963). The measurement of readability. Ames, IA: Iowa State University Press.Klare, G . R . (197 6). A second look at the validity of readability formulas. Journal of Reading Behavior,8, 129-152.

Klare, G. R . (1984 ). Readability. In P. D . Pearson (E d.), Hand book of reading research (pp. 681-74 4).New York: Longman.Kniffen, J. D., Stevenson, C. R., Klare, G. R., Entin, E. B., & Slaughter, S. L. (1979). Operationalconsequences of literacy gap (AFHRL Tech. Rep. No. 79-22). Lowry Air Force Base, CO:Technical Training Division, Air Force Human Resources Laboratory.Lawton, J. T., & Wanska, S. K. (1977). Advance organizers as a teaching strategy: A reply to Barnesand Clawson. Review of Educational Research, 47, 233-244 .Loman, N. L., & Mayer, R. E. (1983). Signaling techniques that increase the understandability ofexpository prose. Journal of Educational Psychology, 75, 402-412 .Luftig, R. L., & Greeson, L. E. (1983). Effects of structural importance and idea saliency on discourserecall of mentally retarded and nonretarded pupils. American Journal of Mental Deficiency, 87,414- 421 .Mayer, R. E. (1979). Can advance organizers influence meaningful learning? Review of EducationalResearch, 49, 3 71 -3 8 3 .McCarthy, L. (1987). A stranger in strange lands: A college student writing across the curriculum.Research in the Teaching of English, 21, 2 3 3 -2 6 5 .McGee, L. (1982). Awareness of text structure: Effects on children's recall of expository text. ReadingResearch Quarterly, 17, 5 8 1 -5 9 0 .McLean, J. I. (1986). Forma tive evaluation of the effectiveness of texts. (ERIC Document ReproductionService No. ED 276 763).Melton, R. F. (1978). Resolution of conflicting claims concerning the effect of behavioral objectiveson student learning. Review of Educational Research, 48, 2 9 1 -3 0 2 .Meyer, B. J. F. (1975). The organization of prose and its effects on memory. Amsterdam: NorthHolland.Meyer, B. J. F. (1977). What is remembered from prose: A function of passage structure. In R. O.Freedle (Ed.), Discourse production and comprehension (Vol. 1, pp. 307-336). Norwood, NJ:Ablex.Meyer, B. J. F., (1984). Text dimensions and cognitive processing. In H. Mandl, N. L. Stein, & T.Trabasso (Eds.), Learning and comprehension of text (pp. 3-51). Hillsdale, NJ: Erlbaum.

Meyer, B. J. F., Brandt, D. M., & Bluth, G. J. (1980). Use of top-level structure in text: Key forreading comprehension of ninth-grade students. Reading Research Quarterly, 16, 7 2 - 1 0 3 .Meyer, B. J. F., & Freedle, R. O. (1984). Effects of discourse type on recall. American EducationalResearch Journal, 21, 121-143 .Miller, S. M. (1988). Collaborative learning in secondary school discussion of expository texts (Doc-toral dissertation, University of Pittsburgh, 1988). (University Microfilms No. 89-05, 229).Moore, D. W., Readence, J. E., & Rickelman, R. J. (1983). An historical exploration of content areareading instruction. Reading Research Quarterly, 18, 4 1 9 -4 3 8 .



27/28


National Assessment Educational Progress. (1981). Reading, thinking, and writing: Results from the1979 -80 National Assessment of Reading and Literature (Rep. N o. 11-1-01). Denver, CO : Educa-tion Commission of the States.

National Assessment of E ducational Progre ss. ( 1987, Octobe r). Literature and U .S. history (EducationalTesting Service Report No. 17-HL-01). Princeton, NJ: Educational Testing Service.Nelson, J. (1990). This was an easy assignment: Examining how students interpret academic writingtasks. Research in the Teaching of English, 24, 362-396 .O'Hear, M. F., Ramsey, R. N., & Pherson, V. E. (1987). Location of main ideas in English compositiontexts. Research in the Teaching of English, 21, 318-326 .Pearson, P. D. (1975). The effects of grammatical complexity on children's comprehension, recall, andconception of certain semantic relations. Reading Research Quarterly, 10, 155-192.Phifer, S. J., McNickle, B., Ronning, R. R., & Glover, J. A. (1983). The effects of details on therecall of major ideas in text. Journal of Reading Behavior, 15, 19-30 .Rede r, L. M , & Anderso n, J. R. (198 0). A comparison of texts and their summ aries: Mem orialconsequences. Journal of Verbal Learning and V erbal Behavior, 19, 121-134.Redish, J. C. (1979). Readability. Washington, DC: Document Design Center, American Institutes forResearch.Richgels, D. J., M cGee, L. M ., Lomax, R. G ., & Sheard, C . (1987). Awareness of four text structures:Effects on recall. Reading Research Quarterly, 22, 177-196.Rickards, J. P. (1979). Adjunct postquestions in text: A critical review of methods and processes.

Review of Educational Research, 49, 181-196.Rickards, J. P., & Di Vesta, F. J. (1974). Type and frequency of questions in processing textualmaterial. Journal of Educational Psychology, 66, 354-362 .Roen, D. H., & Pich, G. L. (1984). The effects of selected text-forming structures on collegefreshmen's comprehension of expository prose. Research in the Teaching of English, 18, 8 - 2 5 .Roller, C. M. (1990). The interaction between knowledge and structure variables in the processing ofexpository prose. Reading Research Quarterly, 25, 7 9 - 8 9 .Sagaria, S. D., & Di Vesta, F. I. (1978). Learner expectations induced by adjunct questions andthe retrieval of intentional and incidental information. Journal of Educational Psychology, 70,2 8 0 -2 8 8 .Schriver, K. A. (1986). Designing computer docum entation: A review of the relevant literature (hard-copy, online, general application) (Tech. Rep. No. 31). Pittsburgh, PA: Carnegie Mellon Univer-sity, Communication Design Center.Schumacher, G. M., & Nash, J. G. (1991). Conceptualizing and measuring knowledge change due towriting. Research in the Teaching of English, 25, 6 7 - 9 6 .Slater, W. H., Graves, M. F., & Pich, G. L. (1985). Effects of structural organizers on ninth-gradestudents' comprehension and recall of four patterns of expository text. Reading Research Quar-terly, 20, 189-202.

Slater, W. H., Graves, M. F., Scott, S. B., & Redd-Boyd, T. M. (1988). Discourse structure andcollege freshmen's recall and production of expository text. Research in the Teaching of En glish,22 , 4 5 - 6 1 .Spiro, R. J. (1975). Inferential reconstruction in memo ry for connected discourse (Tech. Rep. No. 2).Champaign, IL: University of Illinois, Laboratory for Cognitive Studies in Education. (ERICDocument Reproduction Service No. ED 136 187).

Spiro, R. J. (1980). Constructive processes in prose comprehension and recall. In R. J. Spiro, B. C.Bruce, & W. F. Brewer (Eds.), Theoretical issues in reading comprehension (pp. 245-278).Hillsdale, NJ: Erlbaum.Spyridakis, J. H., & Standal, T. C. (1987). Signals in expository prose: Effects on reading. Reading

Research Quarterly, 22, 2 8 5 -2 9 8 .Swaney, J. H., Janik, C. J., Bond, S. J., & Hayes, J. R. (1981). Editing for comprehension: Improving



28/28


the process th

review of research in revising instructional text

Documents