The Florida State UniversityDigiNole Commons

Electronic Theses, Treatises and Dissertations The Graduate School

11-13-2009

Linguistic Feature Development in ElementaryWriting: Analysis of Microstructure andMacrostructure in a Narrative and an ExpositoryGenreShannon S. Hall-MillsFlorida State University

Follow this and additional works at: http://diginole.lib.fsu.edu/etd

This Dissertation - Open Access is brought to you for free and open access by the The Graduate School at DigiNole Commons. It has been accepted forinclusion in Electronic Theses, Treatises and Dissertations by an authorized administrator of DigiNole Commons. For more information, please contactlib-ir@fsu.edu.

Recommended CitationHall-Mills, Shannon S., "Linguistic Feature Development in Elementary Writing: Analysis of Microstructure and Macrostructure in aNarrative and an Expository Genre" (2009). Electronic Theses, Treatises and Dissertations. Paper 4317.

THE FLORIDA STATE UNIVERSITY

COLLEGE OF COMMUNICATION AND INFORMATION

LINGUISTIC FEATURE DEVELOPMENT IN ELEMENTARY WRITING:

ANALYSIS OF MICROSTRUCTURE AND MACROSTRUCTURE

FEATURES IN A NARRATIVE AND AN EXPOSITORY GENRE

By

SHANNON S. HALL-MILLS

A Dissertation submitted to the School of Communication Science and Disorders

in partial fulfillment of the requirements for the degree of

Doctor of Philosophy

Degree Awarded: Spring Semester, 2010

Copyright © 2010 Shannon S. Hall-Mills

All Rights Reserve

The members of the committee approve the dissertation of Shannon S. Hall-Mills

defended on November 13, 2009.

__________________________________ Kenn Apel Professor Directing Dissertation

__________________________________ Barbara Foorman University Representative

__________________________________ Lisa Scott Committee Member

__________________________________

Shurita Thomas-Tate Committee Member

Approved: ____________________________________________________ Juliann Woods, Director, School of Communication Science and Disorders ____________________________________________________ Lawrence C. Dennis, Dean, College of Communication and Information

The Graduate School has verified and approved the above-named committee

members.

ii

Dedicated to: My family

for their love and support throughout this process

and for teaching me to value education.

iii

ACKNOWLEDGEMENTS

This dissertation is dedicated to my family, especially my parents, husband, and sister for their limitless love and steadfast support. Many thanks go to my major professor, Dr. Kenn Apel, for sharing his expertise, demonstrating his passion for research, teaching, and service, and guiding me through this process with patience and enthusiastic encouragement. I am forever grateful to my entire committee, including Dr. Apel, Dr. Foorman, Dr. Scott, and Dr. Thomas-Tate for their interest and expertise devoted to enhancing this project. I appreciate the mentorship of Drs. Howard Goldstein, Juliann Woods, Amy Wetherby, and Barbara Palmer throughout my doctoral education. I am grateful to Drs. Michelle Bourgeois and Julie Stierwalt for their mentorship throughout my teaching endeavors. Additionally, I owe thanks to several individuals from Volusia County Schools and Florida Department of Education for their role in my education (you know who you are!). I am thankful to the students and teachers that participated in this project and give our work meaning and value. I could not have done this study without the support of doctoral students Elizabeth and Danielle, and am thankful for their assistance with data collection and numerous logistics. The education field will continue to benefit over the long haul from your enthusiasm and commitment to children. I am indebted to graduate research assistants Sherilyn, Jennifer, and Tiffany for their dedication and long hours coding SALT files, and for our well-timed chats. I appreciate your interest in written language development, and hope this experience was something you can build on. Also, thanks to their classmate Sara, who consistently offered support in juggling the course I was teaching while analyzing the data. The future is bright for all four of you! And not far behind are four wonderful undergraduate volunteers, Liz, Amanda, Mary, and Katie, whose voluntary involvement in the post hoc analysis is greatly appreciated. Your enthusiasm for the field is encouraging! Numerous people supported my doctoral endeavors financially. My sincerest gratitude goes to Drs. Goldstein and Woods for assistantship funding through their language and literacy leadership training grant. Furthermore, the Kappa Kappa Gamma Foundation graciously provided financial support through scholarships, and I am grateful to the local chapter and alumni as well for their moral support. Many thanks go to the FSU Congress of Graduate Students for presentation grants for multiple conferences, and to the Graduate School, for the FSU Dissertation Research Grant that afforded the researcher version of SALT and the GRADE measure. To my fellow doc students: you made the journey an absolute pleasure! I can hardly wait to see what you do next! Your passion for what you do, and your faith and perseverance are truly inspirational. I’m especially grateful for Alisha, who strengthens my faith and has been a constant supporter, Elizabeth, who helps me see the forest through the trees and knows precisely when happy hour is order, Rachel, for her never ending smile and contagious laughter, Naomi for her work ethic, and David, for his fresh perspective. Thanks also to Danielle, Lori and Janine specifically for their helpful feedback on the presentation. From early on in the program, Jessika, Kimberly, Leila, and Kerry have been fantastic cheerleaders. I also appreciate friends outside of the program who didn't give up on me, especially Jennifer and Christa. Last, but certainly not least, thank you to each teacher in my past who took the time to support my learning and cheer me on. The world is a better place because of you!

iv

TABLE OF CONTENTS List of Tables ................................................................................. vii List of Figures ................................................................................ viii Abstract .................................................................................... ix 1. Introduction .............................................................................. 1 Written Product ......................................................................... 2 Microstructure ................................................................... 3 Macrostructure.................................................................. 7 Microstructure and Macrostructure in a Single Genre....... 11 Microstructure and Macrostructure in Multiple Genres ..... 15 Research Questions and Hypotheses....................................... 16 2. Method .................................................................................... 18 Participants ............................................................................... 18 Measures .................................................................................. 19 Procedure ................................................................................ 22 Data Analysis............................................................................ 22 Dependent Measures ....................................................... 22 Research Assistant Training ..................................................... 25 Inter-Rater Reliability ............................................................... 26 Research Design ...................................................................... 27 Power Analysis ......................................................................... 27 3. Results ……… ........................................................................... 29 Preliminary Analyses ……………………………………………… 29 Multiple Analyses of Covariance (MANCOVA) ………………… 33 Relations among Measures of Microstructure and Macrostructure 39 Findings Related to Ethnicity, Gender and SES ...................... 40 Post Hoc Analysis .................................................................... 41 4. Discussion …………………………………………………………… 52 Effect of Grade Level on Microstructure ………………………… 52 Effect of Genre on Microstructure ……………………………….. 55 Effect of Grade Level on Macrostructure ……………………….. 55 Effect of Genre on Macrostructure ………………………………. 58 Relations among Measures of Microstructure and Macrostructure 59 Effects of Ethnicity, Gender, and SES …………………………... 60

v

Limitations and Future Research …………………………………. 62 Educational Implications …………………………………………… 64 Conclusion ………………..………………………………………… 65 APPENDICES ................................................................................ 66 A Consent Form and IRB Approval ...................................... 66 B Writing Instructions and Prompts ...................................... 67 C SALT Protocol for Microstructure Variables ………………. 68 D Protocol for Macrostructure Variables …………………….. 73 REFERENCES .............................................................................. 77 BIOGRAPHICAL SKETCH............................................................. 83

vi

LIST OF TABLES

Table 1: Demographic Characteristics of Participants by Grade Level ...... 19 Table 2: Means and Standard Deviations for Independent Measures ....... 21 Table 3: Dependent Writing Variables ....................................................... 25 Table 4: Cohen’s Kappa Coefficients and Percent Agreement .................. 27 Table 5: Four Factor Solution of Microstructure......................................... 42 Table 6: Factor Loadings ........................................................................... 43 Table 7: One Factor Solution of Macrostructure ........................................ 43 Table 8: Factor Loadings ........................................................................... 44 Table 9: Factors and Respective Dependent Variables ............................ 44 Table 10: Descriptive Statistics for Dependent Measures, Narrative........ 45 Table 11: Descriptive Statistics for Dependent Measures, Expository....... 46 Table 12: Adjusted Means and Standard Deviations; Narrative ................ 47 Table 13: Adjusted Means and Standard Deviations; Expository ............. 47 Table 14: Overall and Grade Level Correlations ....................................... 48 Table 15: Correlation Matrices; Narrative and Expository.......................... 48 Table 16: Grade Level Correlations Matrices; Narrative and Expository ... 49 Table 17: Descriptive Statistics for 5 Writing Factors by Ethnicity, Gender, and SES .................................................................................................... 51

vii

LIST OF FIGURES

Figure 1: Productivity; Narrative and Expository ........................................ 37 Figure 2: Grammatical Complexity; Narrative and Expository.................... 38 Figure 3: Grammatical Accuracy; Narrative and Expository....................... 38

Figure 4: Lexical Diversity; Narrative and Expository................................. 39

Figure 5: Macrostructure; Narrative and Expository................................... 39

viii

ABSTRACT

The purpose of this study was to examine multiple dimensions of written language

produced by eighty-nine children in grades 2, 3, and 4 in narrative and expository writing

samples. Two written composition samples were collected from students exhibiting typical

development in second, third, and fourth grades using one narrative and one expository writing

prompt via a scripted, generated elicitation method. Additionally, participants completed group-

administered, norm-referenced measures of receptive vocabulary, word level reading, and

reading comprehension. The writing samples were transcribed into Systematic Analysis of

Language Transcripts (SALT; Miller & Chapman, 2005), coded, and analyzed for developmental

progression of linguistic elements represented by the five factors of productivity, grammatical

complexity, grammatical accuracy, lexical diversity, and macrostructure. Reading

comprehension scores were used as covariates in the multivariate analyses of variance.

Results indicated that levels of productivity and macrostructure increased steadily with

age. Across the narrative and expository samples examined, levels of productivity were highly

correlated and nearly equivalent within each grade, whereas a trend was noted for levels of

macrostructure in the expository genre to increase more sharply from second to third grade than

in the narrative genre. There was a grade effect for grammatical complexity in the expository

genre, whereas there were no significant differences between grade levels for narrative

grammatical complexity. Interestingly, the second graders scored higher than the third and

fourth graders on measures of grammatical complexity (especially MLTu) in their expository

samples. Comparison of grammatical complexity levels across genres revealed a small,

negative correlation across all three grade levels. No grade level differences were detected for

grammatical accuracy and lexical diversity in either genre; although, there was a trend for fourth

graders to produce a higher number of grammatical errors than second and third graders.

Students in each grade performed similarly regardless of genre type on measures of

grammatical accuracy and lexical diversity. Relations among measures of microstructure and

macrostructure were revealed between productivity and macrostructure in both genres and

between macrostructure and grammatical accuracy in the expository genre. Inter-correlations of

measures within grade level are discussed. There were no significant effects of ethnicity,

socioeconomic status, or gender on writing outcomes. Interestingly, trade-offs in performance

on certain linguistic features appeared to occur for second and fourth graders.

Results of this study suggest that variables of written microstructure and macrostructure

were sensitive to grade and genre level differences, that productivity (a measure of

microstructure), and macrostructure were related in both genres for all three grade levels, and

ix

that one cannot assume the older students will outperform younger students on all measures.

This latter finding was thought to be due to a trade-off between linguistic and cognitive demands

for second and fourth graders. Consequently, future research needs to establish these trade-off

trends occur in larger samples and examine the effects of different academic contexts (e.g.,

variable elicitation techniques, discourse structures, content specific assignments) on this

phenomenon. The findings of this investigation are discussed in light of grade level standards

for writing and the identification of students with writing difficulties. Multiple suggestions are

presented for educational implications of the results, and specific directions provided for future

research.

x

CHAPTER 1

INTRODUCTION

The arrival of the “information age” illuminates the importance of literacy in a constantly

changing world. Throughout the first decade of the 21st century, we have seen a tremendous increase

in the intensity of literacy demands needed to function within the world, needs that largely were

anticipated a decade prior (Kennedy, 1993). Literacy encompasses both reading and writing

proficiency. To address gaps in students’ reading proficiency, programs such as Reading First (US

Department of Education, 2008) and reports such as Reading Next (Biancarosa & Snow, 2006) have

highlighted the reading difficulties of elementary and adolescent students, summarized relevant

research, and expressed recommendations for the provision of quality reading instruction.

Like reading, writing is central to the quality of education from early childhood to postsecondary

schooling; it is an essential skill for literacy success in school and beyond (Troia, 2009). In the United

States, in an age of standards-based educational reform, writing is used to monitor adequate yearly

progress (AYP) as required by the No Child Left Behind Act (NCLB, 2001), and to determine grade

promotion and high school graduation. Writing proficiency is a significant predictor of reading

performance (Fitzgerald & Shanahan, 2000; Jenkins, Johnson, & Hileman, 2004) and is required for

college entrance and graduation. Furthermore, writing proficiency enables an adult individual to fully

participate in civic life and the economy (Graham & Perin, 2007). Writing is integrated in all aspects of

society for purposes of communication (e.g., medium for conveying knowledge and ideas, exploring self

expression, preserving history, achieving order based on written law, facilitating communication across

distances and time; MacArthur, Graham, & Fitzgerald, 2006). Now more than ever before, writing

proficiency is crucial for obtaining and maintaining employment (Smith, 2000).

More than 30 years ago, Lerner (1976) conjectured that "poor facility in expressing thoughts

through written language is probably the most prevalent disability of the communication skills" (p. 266).

Unfortunately, poor writing performance continues to plague the majority of students in our nation’s

schools. The National Center for Education Statistics (NCES) indicates that the majority of students,

70% of 4th graders (NCES, 2003) and 68% of 8th graders and 76% of 12th graders (NCES, 2007)

performed at or below the basic achievement level in writing based on results of the 2002 and 2006

National Assessment of Educational Progress writing assessment (NCES, 2003, 2007). This

prevalence of writing difficulties has fueled a surge of interest in recent years to further investigate

1

various aspects of writing development and writing proficiency to improve the writing outcomes of

students.

In the past, investigators have employed a variety of measures to study two general aspects of

writing: the writing process, and the writing product. When examining the writing process, researchers

have evaluated children’s ability to plan, generate, and revise text (Graham & Harris, 2003; Nelson &

Van Meter, 2002; Singer & Bashir, 2004). Those who have studied the writing product have examined

compositions for specific linguistic components such as productivity, grammatical complexity, lexical

diversity, text structure, organization, and coherence (Nelson & Van Meter, 2007; Puranik, Lombardino,

& Altmann, 2008; Scott & Windsor, 2000). The purpose of the present paper is to look more specifically

at the current knowledge regarding development of the written product in the elementary years.

Written Product

When examining the written product, investigators have analyzed children’s writing generated

under certain variations of the rhetorical task (Singer & Bashir, 2004). For example, children have been

instructed to generate text using a writing prompt that is expected to elicit a particular text structure for

a given discourse genre (Scott, 2009). Discourse genres represent different forms and styles of writing

and reflect a range of purposes and contexts for writing (Graham & Harris, 2003; Graham & Perin,

2007). In the school environment, narrative and expository genres are the most commonly encountered

discourse genres (Donovan & Smolkin, 2006), and therefore, the most commonly used writing prompts

in the elementary grades are intended to elicit either narrative or expository texts. Narrative discourse

involves telling a story, often about personal events or other life experiences (e.g., novels, personal

letters, and short stories). In contrast, expository discourse involves conveying facts or describing

procedures, sharing basic information, relating cause-effect relationships, or arguing a point of view

(e.g., essays, editorials). The ability to write proficiently in both narrative and expository genres is linked

to academic success (Nelson, Bahr, & Van Meter, 2004; Singer, 2007).

Knowledge of discourse genres is acquired in a developmental progression and is related to

reading comprehension and writing achievement (Englert & Thomas, 1987). Awareness and use of

narrative discourse in written language typically develops first, often through storytelling experiences

(Nelson, et al. 2004). Compared to narrative discourse, the structure of expository discourse is typically

mastered later in the school years and, as a consequence, is more difficult to produce and comprehend

for many students (Berman & Verhoeven, 2002). Much of the recent research regarding discourse

genres in written language has centered on text comprehension; in contrast, fewer studies have

focused on text production (i.e., writing). Furthermore, when researchers have examined linguistic

features at the discourse level in written language, their investigations often have been limited to

narrative discourse. There is a need, then, to examine students’ writing skills across additional

2

discourse genres, such as expository genre, especially considering that by the 4th grade, 60% of writing

assignments are expository in nature (Graham & Perin, 2007; Persky et al., 2003).

When examining the written product in different discourse genres, investigators have conducted

analyses of elements of microstructure and macrostructure. Analysis of linguistic elements at the

microstructural and macrostructural levels of a text have great potential for capturing the development

of linguistic features, and describing the challenges faced by students who struggle with the generation

of written language.

Microstructure

Analysis of elements of microstructure in a written product can occur on multiple levels,

including examination of linguistic elements at the word, sentence, and/or discourse levels.

Microstructure analysis generally examines a writer’s conveyance of meaning at these levels and

typically includes measures of productivity (e.g., number of words, T-units, or ideas), grammatical

complexity (e.g., mean length of T-unit, clause density), and lexical diversity (e.g., type-token ratio,

number of different words) (Nelson et al., 2004; Puranik, Lombardino, & Altmann, 2007, 2008). T-units,

or “minimal terminable syntactic units” (Hunt 1966), are defined as one main clause and any

subordinate clauses and are the most common unit of segmentation for written language transcripts.

Narrative Microstructure

There is a paucity of investigations focusing solely on the development of elements of

microstructure in products written in a narrative genre. Houck and Billingsley (1989) analyzed the

development of microstructure in narrative samples of 16 students with typical development in grades

4, 8, and 11. Participants were allowed 20 minutes to write a story about a trip. Their written narratives

were analyzed for various measures of productivity, grammatical complexity, and lexical density (e.g.,

number of words, sentences, words per sentence, and sentence fragments, words with more than 7

letters, and T-units, mean morphemes per T-unit), percentage of correct capitalization, and percentage

of correct spelling. Results indicated that the 4th graders in the sample produced an average of 152.75

words, 10.63 sentences, 16.02 words per sentence, 13.44 T-units, 12.54 mean morphemes per T-unit,

1.13 sentence fragments, 10.06 words with 7 or more letters, 91.5% correct capitalization, and 94.9%

correct spelling. Significant grade effects were found for 3 of the 9 variables: number of T-units,

spelling, and lexical density, indicating that increases in productivity, lexical diversity, and spelling

proficiency could be detected among groups at the elementary, middle, and high school levels.

Expository Microstructure

In contrast with the narrative genre, a greater number of previous studies exist that have

examined elements of microstructure in products written in an expository genre. Morris and Crump

(1982) compared the expository writing development of 72 students with typical development ranging in

3

ages from 9 to 15 years on measures of syntactic and vocabulary development. Participants were

instructed to watch a video and subsequently composed an essay. The written products were analyzed

for mean length of T-unit (MLTu), Syntactic Density Score (SDS), type/token ratio (TTR), and

Vocabulary Intensity Index (VII). The SDS was calculated based on a formula that considered the T-unit

length as well as clause length, number of subordinate clauses, embeddings, and verb expansions and

had previously been shown to detect increases in syntactic density between adjacent elementary

grades (Blair & Crump, 1984). The Vocabulary Intensity score was calculated using the Vocabulary

Intensity computer program (Kidder, 1974). The results indicated that MLTu, TTR, and SDS increased

consistently across the age levels. Eighteen students with typical development at age level 9.0-10.5

years (similar to students in grades 4 and 5 of other investigations), produced an average MLTu of

7.45, mean SDS of 2.02, and mean TTR of 3.27. The mean scores on the Vocabulary Intensity Index

were not reported and no differences were found between age levels on this measure). These findings

indicated that commonly employed measures such as MLTu and TTR were sensitive to differences in

expository grammatical complexity and lexical diversity between successive age levels, beginning with

age 9 years and up, and that SDS provided additional qualitative information about the quality of syntax

reflected in the written expository products of students in the varying age level groups.

In a later study conducted by Puranik, Lombardino, and Altmann (2008), the development of the

expository writing of 120 children exhibiting typical development in grades 3 (mean age = 8.7 yrs.), 4

(mean age = 9.7 yrs.), 5 (mean age = 10.8 yrs.), and 6 (mean age = 11.7 yrs.) was examined. One

writing sample per participant was collected using an expository text-retelling paradigm (to reduce

working memory load). Participants were instructed to listen to an expository passage and then write

what they remembered about the passage. No time restrictions were enforced, but the majority of

participants completed their writing in 10 minutes. The T-unit was used as the unit of segmentation. In

perhaps the most in-depth analysis of expository microstructure development to date, Puranik

examined 13 variables of microstructure at the word, T-unit, sentences, and discourse levels, including

number of words, ideas, T-units, clauses, sentences, and different words, MLTu, clause density, errors

per T-unit, percentage of grammatically correct sentences, sentence complexity, percentage of spelling

errors, and writing conventions.

Puranik et al's results indicated that measures of productivity and grammatical complexity

increased with age. A significant multivariate effect of grade and significant main effect for total number

of words were indicated, and pairwise comparisons indicated significant differences in total number of

words between participants in grades 3 and 4, (d=1.09, p<.001). Significant differences were evident

between the 3rd and 4th grade groups for the variables of total words, total ideas, number of T-units,

number of clauses, number of sentences, sentence complexity, and number of different words. The

4

group mean performance of participants in grade 3 was as follows for the 13 microstructure variables:

total words (61.0), total ideas (6.8), number of T-units (6.5), MLTu (9.6), clause density (1.78), number

of clauses (11.2), errors per T-unit (0.30), number of sentences (5.9), % grammatical sentences (73%),

sentence complexity rating (8.8), number of different words (33.8), percentage of spelling errors (7.2%),

writing conventions (88.6). The following descriptive statistics provide the group mean scores for the

participants in the 4th grade group: total words (89.2), total ideas (9.4), number of T-units (8.5), MLTu

(10.5), clause density (1.77), number of clauses (15.0), errors per T-unit (0.21), number of sentences

(7.9), % grammatical sentences (81%), sentence complexity rating (14.7), number of different words

(41.7), percentage of spelling errors (5.5%), writing conventions (90.1). Additionally, results of a factor

analysis confirmed that the 13 microstructure variables examined clustered into 4 dimensions of written

language microstructure: productivity, complexity, accuracy, and mechanics.

Cross Genre Microstructure

Occasionally, investigators have been interested in microstructure performance across more

than one discourse genre. Scott and Windsor (2000) studied the spoken and written language samples

of 60 students, including a group of 20 students with typical development (mean age = 11:5) across two

discourse genres (narrative, expository). Participants were instructed to write a story in response to a

19-minute narrative video, and to write a summary of a 15-minute expository video. The participants

were shown a model paper of expected length and allowed 20 minutes to write. The written samples

were transcribed and coded in SALT with the T-unit as the unit of segmentation, and analyzed for

elements of productivity/fluency (e.g., total number of T-units, words, and time, T-units per minute, and

words per minute), lexical diversity (e.g., number of different words based on first 100 words in the

sample for narrative, first 50 words for expository), grammatical complexity (e.g., words per T-unit,

clauses per T-unit), and grammatical error (e.g., errors per T-unit).

Across the written narrative products, students (mean age = 11:5) exhibiting typical

development (TD) wrote for an average of 23.6 minutes and produced an average of 32.3 T-units, 341

words, 1.4 T-units per minute, 14.2 words per minute, 60.6 different words, 10.4 words per T-unit, 1.94

clauses per T-unit, and .12 errors per T-unit. For the expository writing products, the TD participants

wrote for an average of 21 minutes and produced an average of 18.5 T-units, 216 words, 0.9 T-units

per minute, 10.3 words per minute, 61.6 different words, 12.1 words per T-unit, 1.74 clauses per T-unit,

and .15 errors per T-unit. Significant differences were found between genre means; all 5

productivity/fluency measures showed higher values in written narrative than in written expository

products. Regarding grammatical complexity, the direction of the effect of genre differed; there were

more clauses per T-unit produced in narrative products but more words per T-unit produced in the

expository products. There were no statistically significant genre effects for grammatical error. For all

5

microstructure variables except words per T-unit, the main effects for genre indicated higher levels of

performance in the narrative genre. The authors concluded that more fine-grained analyses of the

influence of genre on clause types are warranted. Differences were not examined across grade levels

or for age effects.

In a cross-linguistic study of 7 languages, including English, comparing four age levels (grades

4, 7, 11, and adult), 2 genres (narrative and expository) and 2 modalities (spoken and written), Berman

and Verhoeven (2002) examined multiple aspects of development of narrative and expository

microstructure. The age range of the 20 English participants in grade 4 was 9-10 years. Participants

were shown a 3-minute video that included scenes of conflict between people. To elicit narrative

writing, the participants were then instructed to write a story about a conflict or a problem they

experienced with someone. To elicit an expository composition, participants were asked to write on the

topic of problems between people and to express their thoughts on the subject, not to write a story.

Participants completed the spoken and written samples over two sessions, with genre order

counterbalanced across groups of participants. The writing samples were produced via a traditional

paper and pencil task, and were subsequently transcribed into a computer data base following standard

Codes for the Human Analysis of Transcripts (CHAT) conventions (MacWhinney, 1995), with the

clause as the segmentation unit. Measures included lexical diversity (vocabulary density, VOCD,

measured by a ratio of word types per token), total number of words, and mean clause length.

Specific means and standard deviations for the 20 English-speaking fourth graders in the study

were not reported. However, visual analysis of graphic illustrations indicates that the mean score for

total words in the narrative and expository genres was approximately 90, and 60 respectively. These

group means for narrative and expository productivity were markedly lower than for the participants in

the Scott and Windsor (2000) investigation. However, the mean age of the participants in Scott and

Windsor was higher (11.5 yrs) than those in the Berman and Verhoeven sample (age range 9-10

years), suggesting age level differences may exist between ages 9-10 and 11 years for both narrative

and expository productivity. Compared to the 4th graders in Puranik’s (2006) study, the mean number of

words in the expository products was lower for the 4th grade English sample in Berman and

Verhoeven’s study (mean = 89.2 and approx. 60, respectively), despite similar elicitation methods.

Regarding mean clause length, the 4th graders in Berman and Verhoeven’s sample produced an

approximate mean of 5.7 clauses in the narrative genre products and 5.25 clauses in the expository

genre products. Main effects across the entire sample for genre and age were indicated for total words,

number of clauses, and vocabulary density, all qualified by significant interactions between genre and

age. These results indicate developmental trends as exhibited in increasing levels of productivity,

grammatical complexity, and lexical diversity across genres (generally favoring the narrative genre) and

6

age levels (greatest levels of performance for oldest participants). However, visual analysis of the

results for the 4th grade English group’s VOCD scores indicate little difference between narrative and

expository lexical density at the 4th grade level (approximately mean VOCD of 50 and 52.5,

respectively). This result can be further explained by the significant interaction effect, for the entire

sample, between genre and age for lexical density. In other words, age differences may play a more

substantial role in one genre than in another for these particular variables.

Like Morris and Crump (1982), and Houck and Billingsley (1989), Berman and Verhoeven

(2002) found that measures of microstructure were sensitive to developmental change across

elementary, middle, and high school age levels. Furthermore, when considering the potential influence

of genre in development of microstructure, the Berman and Verhoeven results indicated little to no

effect of genre on 4th grade narrative and expository lexical diversity, and thus provide further support to

the similar findings of Scott and Windsor (2000), who reported the total number of different words as a

measure of lexical diversity.

In summary, previous investigations have examined the development of microstructure features

in one or more discourse genres. The findings have indicated that measures of productivity,

grammatical complexity, and lexical diversity can be sensitive to age and grade level. Furthermore,

Puranik (2006) detailed how various measures of microstructure cluster into 3-4 dimensions of written

language. This information is collectively important for explaining what children do linguistically with

their written products. However, multiple gaps remain regarding writing development of typically

developing students in grades 2-4. For example, more needs to be known about children's

development of certain microstructure elements (e.g., productivity, grammatical complexity, lexical

diversity) of the written product across multiple genres (e.g., narrative, expository) and between

subsequent grade levels within the elementary years (e.g., comparisons between performance in

grades 2, 3, 4).

Furthermore, there are additional levels of text that can be examined beyond the microstructure.

For example, there is also the knowledge of text structure and organization and coherence of the text

that can provide a different view or perspective on typical writing development in young children in the

2nd through 4th grades.

Macrostructure

In contrast to microstructure analysis, which usually involves comparison of features at the

word, T-unit, sentence and/or discourse levels, macrostructure analysis occurs mainly at the discourse

level (Scott, 2009). With the microstructure as a text base, macrostructure is the “abstract

representation of the global meaning structure…” which represents the “gist” of the text (Sanders &

Schilperood, 2006, p.387). Macrostructure analysis examines a writer’s conveyance of meaning at the

7

discourse level and may include measures of organization, cohesion, and genre-specific text structure.

Elements of macrostructure are often included in qualitative writing analyses, such as in holistic or

analytic scoring systems, or can be depicted quantitatively by counting cohesive ties or genre-specific

text structure elements present in a written product (e.g., counting story grammar elements in a

narrative text, or marking whether an introduction, body, and conclusion are present in an expository

text).

Narrative Macrostructure

Researchers have examined development of elements of macrostructure in products written in a

narrative genre. Laughton and Morris (1989) compared stories generated by 96 students with typical

development in grades 3 through 6 (n=24 per grade; age range 9-12 years) for inclusion of story

grammar elements. Students viewed a filmstrip within their classrooms and were asked to write a story

about the film. No time limits were imposed and the writing was generated using paper and pencil. The

narrative writing samples were scored for presence or absence of the major story components:

exposition (introduction of main character and supporting characters, relationship between characters,

and scene set), complication (defined a problem or conflict), causal and temporal relationships

statements, and resolution (statements focused on solving the problem or achieving the goal).

Results revealed that 54% of students in 3rd and 4th grades produced complete stories. Results

were reported for 9 story grammar components. Percentages for these components for grades 3 and 4,

respectively were: main and other characters = 100% of students in both grades, character relationship

46% (3rd) and 38% (4th), location 50% (3rd), 63% (4th), and time 63% (3rd), 63% (4th). For the

components of complication, the following percentages were obtained: defining the problem 75% (3rd),

63% (4th), causal statements 46% (3rd), 63% (4th), and temporal statements 50% (3rd) and 67% (4th).

Fifty-four percent of both third and fourth graders included the component of resolution. While these

percentages were not compared statistically, the results are visibly suggestive of developmental effects

on the inclusion of story grammar components between these two grade levels. (A general

developmental trend was noted to occur from third to sixth grade for causal and temporal relationships).

It is plausible that a more fine-grained analysis of story grammar structure features, such as through an

analytic scoring technique with multiple levels of performance possible for each variable, would reveal

additional information about the intra-grade development of these features in the written product.

Montague, Maddux, and Dereshiwsky (1990) measured the development of narrative

macrostructure in 36 students with typical development in grades 4/5, 7/8, and 10 (12 participants per

grade level). The researchers compared stories generated under 2 conditions: oral story retell (a story

conforming to canonical story grammar framework) and writing in response to a story starter (a paper

and pencil task). Both tasks were generated individually in a single session in a counterbalanced

8

fashion to control for order effects. No time limits were imposed, and the participants completed both

tasks in a total of 45 minutes. The oral retell samples were scored for presence or absence of 25

propositions from the story. The propositions were categorized into major setting (introducing the

protagonist), initiating event (change in state of affairs requiring protagonist response), attempt (goal-

related actions of protagonist), internal response (affective, emotional responses), direct consequence

(indicating whether goal is attained and any resulting changes), and reaction (character’s feelings,

thoughts related to outcome). The raters also identified the intercategory errors (temporal reversal of

two statements from different categories), intracategory (reversal of two statements within the same

category), and single statement reversal errors. Furthermore the oral retell samples were scored for

substitutions, additions, and deletions of material. The written products produced with a story starter

were scored using two procedures; the first for parsing and categorizing propositions, and the second

for a holistic rating of the cohesion, organization, and episodic structure of the story on a 5 point Likert

scale. Results indicated that no developmental differences existed between or within tasks. However,

the small sample size per grade level (n=12) may have masked potential developmental differences.

Expository Macrostructure

As with the narrative genre, investigators have been interested in the development of expository

macrostructure in children's written products. Englert, Raphael, Anderson, Gregg and Anthony (1989)

measured metacognitive processes and use of expository text structure features in 138 students in

grades 4 and 5 (age range 9-11 years) who were grouped according to their reading achievement

levels (high achievement, low achievement, learning disabilities; 46 students per ability group).

Participants completed 2 expository compositions using 2 different expository sub-genre text structures

(e.g., comparison/contrast, explanation), read and recalled expository texts with the same text structure

as the writing products, and wrote summaries of the expository text they read. No time limits were

imposed for completion of the tasks. Written products were given a primary trait score (degree to which

the product incorporated the required organizational pattern for a specific text structure and appropriate

key words and phrases) and a holistic quality score (degree to which the product was interesting and

effectively communicated a particular text structure form). Products reflecting the explanation

expository text structure were scored for the following traits: introduction, comprehensive sequence of

steps, key words, and adherence to explanation organization (introduction, sensible sequence,

closure). Compare/contrast structure products were scored for the following traits: identification of 2

items for compare/contrast, description of similarities, explanation of differences, use of key words, and

adherence to compare/contrast organization (introduction, similarities/differences, conclusion). The trait

and holistic scores were combined to reflect the product's overall organizational score. Productivity was

rated based on the number of ideas in the product. Multiple analysis of variance (MANOVA) revealed

9

significant effects for group and text structure; performance on organizing compare/contrast expository

compositions was significantly greater than that for explanation products. In contrast, the total number

of ideas was higher for the explanation products. Mean scores were not examined across grade levels

4-5.

Cross Genre Macrostructure

Fewer investigators have been interested in the development of macrostructure features across

more than one discourse genre. In an investigation of the relation between reading performance and

use of cohesion in writing, Cox, Shanahan, and Sulzby (1990) examined cohesion in the narrative and

expository writing products of 48 students in grades 3 and 5. The participants were grouped based on

reading performance (e.g., low achieving readers and high achieving readers). Each student completed

2 narrative and 2 expository writing tasks. Narrative writing was elicited following a discussion of 2 sets

of 3 pictures; expository writing was elicited following discussion of 2 researcher-made expository

articles. The narrative discussion focused on story grammar categories (e.g., setting, event, reaction).

The expository discussion included activation of prior knowledge and focused on the organizational

structure of the articles. The participants completed the writing tasks in groups, counterbalanced for

genre and task. The written products were segmented into T-units and analyzed for appropriate or

inappropriate use of simple coreferential and coclassificatory cohesive ties (e.g., pronoun reference,

use of "the" as a specific determiner, comparatives, demonstratives, ellipsis). The raw score count of

these cohesive ties were divided by total number of T-units for 2 proportional scores: appropriate and

inappropriate cohesive ties. The products also were analyzed for cohesive harmony (another

proportional score) and overall quality using a holistic rating scale.

Results of repeated measures ANOVAs for each genre with grade and reading comprehension

levels as independent variables indicated varying and significant effects of genre, grade, and reading

levels. For example, in both genres, there were significant main effects for grade and reading levels for

appropriate cohesive ties, indicating that fifth grade students used appropriate cohesive ties more

frequently than third graders, and good readers used appropriate cohesive ties more frequently than

poorer readers. (There were no significant interactions between grade and reading levels.) However,

the results differed for harmonic cohesion. In the narrative genre, main effects for grade and reading

levels indicated that 5th graders and stronger readers used cohesive harmony more frequently (no

significant interactions). However, in the expository genre, there was not a significant effect for grade

level, only for reading level, indicating that good readers used cohesive harmony more often than lower

achieving readers, but 5th graders did not do so any more frequently than third graders. Furthermore,

results indicated that in the narrative genre, third graders and poor readers used significantly more

inappropriate cohesive ties (no interaction effects). However, in the expository genre, the only main

10

effect was for reading level, indicating that the poor readers used significantly more inappropriate

cohesive ties, but no difference existed based on grade level alone. Interestingly, a significant

interaction between grade and reading level existed. Post hoc analyses revealed the poor readers in 5th

grade more frequently used inappropriate cohesive ties. These findings collectively demonstrated a link

between reading development and knowledge of cohesion.

Most recently, Crawford, Helwig, and Tindal (2004) examined the written products of 169 and

134 students with typical development in 5th grade and 8th grade respectively produced in narrative,

imaginative, persuasive, and expository genres using a trait scoring system for ideas, organization,

sentence fluency, and conventions. Participants completed two assessment tasks, one lasting 30

minutes, the other occurring over a 3-day period. The trait scores from each assessment task were

compared. The results of an analysis of variance indicated there was no significant effect of discourse

genre on the writing trait performance scores of the participants in 5th grade. Paired t-tests revealed a

significant difference in the individual trait performance of the fifth graders for the traits of sentence

fluency and conventions, favoring the 3-day assessment task. Explicit comparisons of the students'

performance between grade levels were not reported. However, visual analysis of the data suggest a

possible age effect for mean composite trait scores favoring the 8th grade sample, at least for the 3-day

assessment (mean composite trait score for 30-minute assessment = 31.84, 35.83; for 3-day

assessment = 34.76, 35.38; fifth grade and eighth grades respectively).

Crawford et al. (2004) was unique in the number and type of discourse genres within which

student compositions were elicited and analyzed for macrostructure elements. A comprehensive

literature search did not reveal any previous investigations involving this type of cross-genre analysis

with the written products of younger students, such as those in grades 2-4, the primary focus of the

present investigation.

Microstructure and Macrostructure in a Single Genre

In many instances, investigators have sought to document developmental trends across both

microstructure and macrostructure variables within a single genre. Nodine, Barenbaum, and Newcomer

(1985) measured the extent to which 31 children with typical development in grades 5 and 6 (mean age

= 11:6) used story schema in their narrative writing products. Their performance was compared to one

group of 30 students with learning disabilities (LD; mean age = 11:7) and another comparison group of

31 students with reading disabilities (RD; mean age = 11:6). The researchers assessed the children's

macrostructure elements of story schema, cohesion, and microstructure elements of productivity using

the creative writing component of the Diagnostic Achievement Battery (DAB; Newcomer & Curtis,

1984), a standardized test designed to evaluate multiple aspects of writing. Participants were shown a

series of pictures and instructed to write a story about them using paper and pencil. The students were

11

encouraged to plan their writing during the first five minutes and then were allowed 20 minutes to write.

The writing samples were scored in three areas: writing categories (the extent to which a story was

produced; story, story-like, descriptive, expressive), measures of productivity/fluency (total words, mean

length of T-unit), and a measure of cohesion (authors noted whether stories were incoherent,

confusing, or included unclear referent).

Nodine et al’s results indicated that the majority of the participants in the group of students

exhibiting typical development (TD; n = 22) generated compositions that were determined to represent

stories, while fewer students produced story-like (6) and descriptive (3) compositions (0 expressive),

and produced an average of 104.4 words and mean T-unit length of 8.6. Four students generated

writing that was rated as confusing and four students included at least one unclear referent. The

findings suggest that 11-year-old children exhibiting typical development have mastered story schema

well enough to write a short story successfully, meeting the basic requirements. The researchers

discussed the possibility that productivity (microstructure) and use of story schema (macrostructure)

were related; however, direct inferential methods of analysis to this effect (i.e., correlation analyses)

were not reported.

Barenbaum, Newcomer, and Nodine (1987) analyzed the narrative written products of students

in grades 3, 5, and 7 for elements of both microstructure (productivity: total words) and macrostructure

(story categories, composition consistency). Similar to Cox et al. (1990), participants were grouped

based on reading achievement levels (low achieving, typically achieving, and learning disabled). The

typically achieving participant group included 19 third graders (age range = 8-10 years), 19 fifth graders

(age range = 10-12 years), and 17 seventh graders (age range = 12-15 years). Two stories were

elicited per participant using a picture prompt in individual sessions. In both sessions, participants were

encouraged to plan, and for the second story, participants were instructed to draw a picture prior to

writing their story. Stories were classified in the following categories as either a story, primitive story,

action sequence, descriptive, or expressive. Significant differences were noted in the composition

category between tasks, grade levels, and ability group. The written products of the third graders as an

age group, regardless of ability level, included fewer "stories" proportionally than the 5th and 7th grade

groups. Interestingly, the 5th graders produced more "stories" than both the 3rd and 7th grade groups.

Possible explanations for this included the increased knowledge of story schema in the 5th grade

beyond what is typically experienced at the third grade level, or other internal factors such as motivation

to write about topics that may not have been grade-appropriate for the older students. The researchers

concluded that text structure and productivity related to the story category.

In a follow-up investigation to Barenbaum et al. (1987) and using the same participants,

Newcomer, Barenbaum, and Nodine (1988) compared the productivity (total words) and coherence in

12

the oral and written narratives produced by the students. As in Barenbaum et al., stories were classified

as either a story, primitive story, action sequence, descriptive, or expressive. To measure coherence,

written products also were measured for whether they included unclear referents, or were confusing or

incoherent. Results revealed that the typically achieving group across grade levels produced more

stories in the oral modality than in the written, but that there was no difference in story production

between modalities for the other achievement groups. Coherence was better in the oral modality for all

groups. As a general trend, the number of coherence errors increased as the complexity of the story

increased. Regarding productivity, the third graders demonstrated the lowest levels, consistent with

developmental expectations.

Also comparing oral and written narratives, Gillam and Johnston (1992) examined elements of

both microstructure (number and mean length of T-unit, percentage of complex T-units, percentage of

grammatically unacceptable T-units, number of predicate types per T-unit) and macrostructure (number

of connectives per T-unit, cohesion, number of constituents) in the stories produced by 40 students in 4

groups: language/learning impaired, chronological age-matched comparison group, spoken language

age-matched group, and reading-matched group (n=10 per group; age range 9-12 years). Sets of

picture prompts were used to elicit 2 oral and 2 written stories per participant. Participants selected a

picture from a set of three and were instructed to create a story based on the picture they selected. For

each participant, the longest oral and longest written narratives were then transcribed into SALT (length

determined by story constituents). The unit of segmentation was the T-unit. Narratives were examined

for complexity of linguistic form (morphemes per T-unit, number of T-units, percent complex T-units,

number of connectives per T-unit including causal, conditional and temporal connectors), and

underlying content (propositions per T-unit, constituents per story, predicate types per T-unit, and

percent of dyadic constituents). Results indicated that for all groups, written narratives were more

difficult to produce than were oral narratives, as demonstrated by fewer morphemes and prepositions

and increased error rate in the written products. However, for the typically developing participants, their

written products exhibited greater grammatical complexity than their oral narratives. Differences were

not examined across grade levels or for age effects.

Continuing the line of investigations comparing the production of oral and written narratives,

Mackie and Dockrell (2004) examined one written and one spoken narrative produced by 33 children in

three groups: children with specific language impairment (mean age = 11 years), chronological age-

matched peers (CA), and language age-matched peers (LA; mean age = 7.3 years; n=11 per group).

The written products were elicited using a picture prompt in a 30-minute writing session. The writing

samples were analyzed for elements of microstructure (number of words, number of words per minute,

proportion of syntax errors, spelling errors), and macrostructure as measured by the Picture Story

13

Language Test (PLST) scales for Content, Abstract-Concrete (5 levels are used to rate the level of

abstract thought or ideation; quality rating of ideation in the sample). Results revealed that the typically-

developing CA group and the younger group of LA-matched peers produced an average of 91 and 53.2

total words, 8 and 4 words per minute, .02 and .04 proportion of syntax errors, .03 and .16 proportion of

spelling errors, and had a mean Content score (PLST) of 15 and 12.4, respectively. These findings

indicate a developmental effect on performance of the older CA group for greater overall productivity,

fluency, syntactical accuracy, and reduced spelling errors in their narrative written products.

Furthermore, correlations among writing, reading, and oral language measures were examined as well.

For the two groups of typically-developing students (CA and LA groups), no statistically significant

correlations existed between reading, oral language, or writing measures; although, trends were noted

indicating a possible relationship between the content measure and total words produced. These trends

suggest a relationship between measures of macrostructure and microstructure. However, the small

sample size and measures chosen (i.e., sensitivity) may have limited the findings and prohibit further

interpretation for the data derived from the Mackie and Dockrell sample.

In a large-scale longitudinal investigation, Fey, Catts, Proctor-Williams, Tomblin, and Zhang

(2004) analyzed the spoken and written narratives of 538 students produced in grades 2 and 4,

including 238 students exhibiting typical development, for elements of microstructure (lexical diversity:

number of different words; grammatical complexity and accuracy: mean length of C-unit, number of C-

units, clause density, percent grammatical C-units), and macrostructure (content, organization, literate

language sophistication as measured by a quality rating scale). The written products were elicited in

both grades using 2 sets of picture prompts. For all participants, greater gains were noted for the

written narratives than for oral narratives between grades 2 and 4. Some differences were detected in

the writing products between the 2nd and 4th grades. For the typically developing group, their written

products demonstrated a significant and positive increase of 15.82 different words (SD=13.68), but a

negligible growth in MLC-u (mean=1.0, SD=1.83), C-units (mean=2.76, SD=4.06), clause density

(mean = 0.19, SD=0.37), a slight negative decrease (-0.02) in percent grammatical C-units (SD=0.08),

and a negligible change in the narrative quality score (mean=1.62; SD=1.95). Follow up Analyses of

Variance (ANOVAs) confirmed that only one measure revealed a statistically significant difference

between grades (number of different words); and none of the post hoc pairwise comparisons were

statistically significant.

More recently, Nelson and Van Meter (2007) examined microstructure elements (productivity:

number of words, number of T-units, number of sentences; lexical diversity: number of different words;

grammatical complexity and accuracy: sentence correctness and complexity, number of grammatical

errors; spelling: percentage words spelled correctly) and macrostructure (story score for narrative

14

maturity, number and type of conjunctions) in the written narratives of 224 students with typical

development in first through fifth grades (participants per grade level: 1st = 8, 2nd = 54, 3rd = 89, 4th = 42,

5th = 42). Participants were instructed to write a story, and were allowed 50-60 minutes to do so. Written

products were transcribed and coded in the SALT program with the T-unit as the unit of segmentation.

Results indicated grade level effects for total words, total T-units, MLTu, number of different words,

proportion of words spelled correctly, story grammar scores, and number and type of conjunctions. Post

hoc analyses revealed that first and second graders, and fourth and fifth graders clustered together with

similar performance on measures of total words, T-units, number of different words, percentage of

correctly spelled words, and story grammar score, whereas third graders retained their own level of

performance, greater than that of the 1st and 2nd graders, but significantly lower than that of the 4th and

5th graders. Regarding number of T-units, the first through third grade groups were similar to each other

but different from the fourth and fifth graders. By the middle of the second grade, typically developing

students produced stories qualifying halfway between a temporal and a causal sequence. No grade

level differences were found for grammatical error rates. Except for proportion of incomplete complex

sentences, grade effects were noted for grammatical complexity and accuracy, as measured by

proportion of simple incorrect, simple correct, and complex correct sentences. In summary,

developmental changes between grade levels and grade level groups (1/2, 4/5) were noted on multiple

microstructure and macrostructure variables.

Microstructure and Macrostructure in Multiple Genres

A thorough literature search did not reveal any investigations focusing on development of

linguistic features using a combination of microstructure and macrostructure measures across two or

more discourse genres. This is surprising considering the importance of both types of linguistic feature

analysis and the prevalence of various discourse genre in school-age curriculum standards.

Proficient writing is essential for one’s academic, vocational, and civic outcomes. This paper

has reviewed current knowledge about the development of the written product in the elementary years.

More specifically, the development of particular linguistic features can be measured at the

microstructure and macrostructure levels. Many researchers have analyzed children’s written products

for elements of microstructure at the word, sentence, and/or discourse levels, and/or elements of

macrostructure, typically in a narrative and/or expository genre.

The investigations reviewed herein have shown that there is a dearth of studies focusing solely

on the development of narrative microstructure. More commonly, researchers have focused on either

the development of expository microstructure, narrative macrostructure, or some combination thereof.

The effect of age and/or grade level on the development of the written product has been documented

between grade-levels (elementary, middle, high), grade level groups (grades 5/6, 4/5, 2/3) and through

15

some subsequent grade comparisons (grades 5 to 6, 4 to 5, 3 to 4). The development of various

linguistic features at the microstructure (productivity, grammatical complexity, lexical diversity) and/or

macrostructure (organization, coherence, text structure features) levels has been documented, but in a

somewhat incongruous manner. In other words, the majority of the investigations reviewed presently

were limited by a variety of factors, including examination of a small set of variables, inclusion of small

sample sizes, analysis on only one level of the written product (microstructure vs. macrostructure),

analysis of written product development in one discourse genre, altogether lacking a truly

comprehensive examination of these linguistic features and cross-genre comparisons for students in

the mid-elementary grades (grades 2 through 4). Additionally, the literature hints at the notion of a

relation between the development of microstructure to that of macrostructure, and this idea makes

sense in light of what is known about linguistic development in general. However, as few people have

explored this relation systematically across multiple genres, such a question warrants further

exploration.

In summary, examining the linguistic elements of microstructure and macrostructure of the

written product is important to gain a better understanding of the development of linguistic features

among typically developing students. Such developmental information can be used to further refine

assessment measures and instructional procedures, and to better define the factors that place students

at risk for writing difficulties.

Research Questions and Hypotheses

The current literature documents developmental changes that occur in the microstructure

element of productivity and macrostructure elements of text structure maturity and coherence among

individual grade levels, grade level groups, and education levels (elementary, middle, high, adulthood).

However, results are less clear regarding developmental changes between elementary grade levels for

additional microstructure elements (grammatical complexity, lexical diversity) and macrostructure

elements across discourse genres.

For the student to become a proficient writer, elements of microstructure and macrostructure

must be incorporated in written text, an accomplishment that requires multiple processes be

coordinated in concert. Because writing performance relies on each of these sets of linguistic

knowledge, both levels of analyses should be included in a comprehensive examination of written

language. Additionally, analysis of the written product generated in different discourse genres will yield

additional information about writing development in different discourse genres. Despite this, there is a

dearth of investigations about writing development in the elementary years, specifically 2nd through 4th

grade, that measure both microstructure and macrostructure across two or more discourse genres

within the same population.

16

The present investigation aims to address some of these gaps in the literature in an attempt to

build the knowledge base about development of microstructure and macrostructure in two genres

during the elementary grade years. Such information will have clinical and educational implications for

writing assessment and instruction for students experiencing writing difficulties.

Specifically, this investigation seeks to document the progression of linguistic elements of

microstructure and macrostructure that students with typical development in 2nd, 3rd, and 4th grades use

in their written narrative and expository compositions. The following research questions guided the

present study:

1. Are there differences among grades and between genres in linguistic microstructure

elements?

It was hypothesized that statistically significant differences would be detected between grade levels

on microstructure measures of productivity and grammatical complexity in both narrative and expository

genres. It was anticipated that performance on microstructure measures would be greater in the

narrative genre, particularly for students in the second, and possibly the third, grades who generally

possess less knowledge of and experience with the expository genre.

2. Are there differences among grades and between genres in macrostructure elements?

Differences were expected between grade levels on variables of organization, coherence, and text

structure for both genres, with the oldest students (4th graders) demonstrating the highest levels of

performance in this regard. The potential genre effects were more difficult to anticipate given the range

of findings in previous investigations. However, it was anticipated that performance on macrostructure

measures would be similar in both genres for the oldest students (4th grade), if the assumption held

true that experience and knowledge of various text structures and text cohesion increases with age.

3. To what degree is development of microstructure elements related to development of

macrostructure elements?

Few researchers have examined this potential relation directly. It was expected that development of

these elements would be related at least in the sense that increased productivity (more words per

written product) allows more opportunities for a writer to incorporate the necessary text structure

elements and genre-specific organizational structure. However, cohesion is accomplished in the written

products of more experienced writers despite overall text length.

17

CHAPTER TWO

METHOD

Participants

Participants were selected from a public elementary school located in a mid-sized city in north

Florida, whose student body is purported to be representative of the state of Florida. The school-wide

student demographics were: gender, 50.4% female, 49.6% male; ethnicity, 56.86% White, 24.54%

African-American, 10.81% Hispanic, 2.46% Asian, 0.56% Native American Indian, 4.74% Multicultural.

The free and reduced lunch rate for the elementary school was 24.1%. The following demographic

information was collected for each participant: gender, age, ethnicity, status in exceptional student

education (if applicable) and whether the student was a recipient of free/reduced lunch (as a measure

of socioeconomic status).

Participants were recruited in conjunction with a larger investigation examining an experimental

spelling intervention. Approval was obtained from the Florida State University Institutional Review

Board (IRB) for the procedures and consent forms for this study (see Appendix A). Additionally,

permission to conduct research was obtained from the school. Consent forms were sent home to all

second, third, and fourth grade students. Participants had to be monolingual English-speaking, enrolled

in general education, with no history of sensory impairments as determined by school records.

Consultation between the PI and research director at the school confirmed whether participants with

parental consent met the inclusionary criteria. Writing samples were obtained from and group

administered measures were conducted with all participants who returned a signed parental consent

form. Attempts were made to recruit equal numbers of male and female students, and for the sample to

be ethnically- and socioeconomically-diverse.

A total of 93 participants were recruited for the spelling intervention study and writing samples

were collected from 89 participants. Four of the consented participants did not complete the writing

samples for the present study and were therefore excluded. The final sample for the present

investigation (n = 89) included 37 males (41.6%), and 52 females (58.4%). The participants ranged in

age from 7 years, 0 months to 10 years, 11 months (M = 8 years, 6 months; SD = 10.9 months). The

participants represented a range of ethnic backgrounds, including 55% Caucasian, 20.2% African

American, 11.2% Hispanic, 3.4% Asian American, 7.9% multiethnic, and 2.2% unreported ethnic

backgrounds. School records indicated that 13 students in the sample were receiving support services

in special education. However, no specific information was provided regarding the students' primary

exceptionalities or details regarding services received. All of the students were enrolled in general

education. Examination of the descriptive statistics on these students' performance on the independent

18

measures indicated mean scores within the average range (Group Reading Assessment and

Diagnostic Evaluation, Reading Comprehension Composite M = 90.5, SD = 8.51; PPVT-IV M = 101.10,

SD = 14.14). Demographic characteristics are depicted per grade level in Table 1.

Table 1

Demographic Characteristics of Participants by Grade Level.

Characteristic Grade Level 2 3 4 Total n Age (M, SD) 7:9 (7.35) 8:5 (3.52) 9:8 (5.77) -- Gender Male 17 (61%) 14 (42%) 12 (43%) 43 Female 11 (39%) 19 (58%) 16 (57%) 46 Ethnicity Caucasian 16 (57%) 20 (61%) 13 (46%) 49 African American 4 (14%) 5 (15%) 9 (32%) 18 Asian American 2 (7%) 1 (3%) 0 (%) 3 Hispanic 3 (11%) 5 (15%) 2 (7%) 10 Other 3 (11%) 2 (6%) 4 (14%) 9 Free/Reduced Lunch Receiving 5 (18%) 8 (24%) 10 (36%) 23 Not Receiving 23 (82%) 25 (76%) 18 (64%) 66 Number of participants per grade level 28 33 28 Note. Standard deviation for age given in parentheses, measured in months. Total sample size = 89.

Measures

Reading

The Group Reading Assessment and Diagnostic Evaluation (GRADE; Williams, 2001) was

administered to obtain participants' reading levels. The GRADE is a norm-referenced, research-based

reading assessment which may be administered in groups. In general, the GRADE measures individual

reading skills in the areas of comprehension, vocabulary, and oral language. Levels 2, 3, and 4, which

correspond to grade levels 2, 3, and 4, were administered to participants within large groups (e.g.,

classrooms). The grade-level appropriate versions of the subtests of Word Reading, Sentence

Comprehension, and Passage Comprehension were administered and standard scores obtained. For

Levels 2 and 3, Word Reading was administered as well. The GRADE measures were scored

according to standardized test protocol. The test was used to corroborate report of participants’ general

19

reading skills within typical limits. The GRADE measure was selected as it allowed direct comparison of

student performance from one grade to another (Moore-Brown, Montgomery, Bielinski, & Shubin,

2005).

The GRADE was standardized using a sample of 33,432 students from 46 states in preschool

through postsecondary grades. According to the test manual, internal consistency of the GRADE was

determined for each subtest, composite, and test level using coefficient alpha and split-half methods

based on classical test theory, and ranged from .95 to .99; thus indicating high levels of internal

reliability for each form, level, and grade-enrollment group. Test-retest reliability coefficients for levels 2

through 4 ranged from .89 to .98. Concurrent validity was established at .71 with the Iowa Test of Basic

Skills (ITBS), .82 to .87 with the California Achievement Test (CAT), and .86 to .90 with the Gates-

MacGinitie Reading Tests (Gates).

Vocabulary

In addition to the GRADE measure, the Peabody Picture Vocabulary Test - Fourth Edition

(PPVT-IV; Dunn & Dunn, 2007) was administered as part of a battery of assessments to ascertain

participants’ receptive vocabulary levels, and to corroborate teacher report of receptive language skills

within typical limits. The PPVT-IV is a measure commonly utilized in language and literacy research.

Internal consistency of the PPVT-IV by age and grade is .94 to .95. Test-retest reliability by age is .93.

The PPVT-IV has average correlations of .72 with the CELF-4 Core Language scale, .71 with the

CELF-4 Receptive Language scale, and .70 with the CELF-4 Expressive Language Scale. Additionally

the PPVT-IV has an average correlation of .60 with the GRADE Total Test Score for levels 2-4.

GRADE and PPVT-IV scores for the participants by grade level are reported in Table 2. One-

way analysis of variance (ANOVA) indicated that the mean group standard scores for the PPVT-IV did

not significantly differ between grade levels, F(2,86) = 2.413, p>.05, partial η2 = .07. ANOVA results for

comparison of grade level means for the GRADE Comprehension Composite scores indicated a

significant main effect for grade level, F(2,86) = 5.86, p=.004, partial η2 = .12. Follow-up tests to

evaluate pairwise differences among the means indicated that there were no significant differences

between second and fourth grades, but there was a significant difference between third and fourth

grades, favoring the third grade group, (mean difference = 9.65, p = .003). The 95% confidence

intervals for the pairwise differences were -9.95 to .364 for the second grade-third grade comparison, -

.57 to 13.57 for the second grade-fourth grade comparison, and 2.85 to 16.45 for the third grade-fourth

grade comparison. Concerns regarding the difference on GRADE Comprehension Composite scores

between the third and fourth grade groups were allayed upon the consideration that the fourth grade

scores, while slightly lower than third grade, still fell well within the average range (M = 95.90).

Examination of the fourth grade scores indicated a small group of scores clustered at the lower portion

20

of the average range, thereby influencing the group mean and standard deviation. Therefore, it was

concluded that the GRADE scores fulfilled their main purpose in the present investigation, to verify

reading scores within normal limits across all three grade levels, and could also serve a second

purpose as a covariate in the main analyses. Previous literature supports the choice of reading ability

as a covariate in studies examining written microstructure and macrostructure (Cox et al, 1990;

Montague et al, 1990).

Table 2. Means and standard deviations for independent measures. Grades 2 3 4 Vocabulary

PPVT-IV Raw score 136.86 (11.61) 144.70 (17.43) 150.12 (14.82)

PPVT-IV SS 110.68 (9.17) 109.03 (14.31) 102.61 (12.42)

Reading (GRADE measure)

Word Reading raw score 26.18 (2.51) 28.82 (1.36) N/A

Sentence Comprehension raw 14.11 (3.94) 17.55 (1.70) 12.46 (3.82)

Passage Comprehension raw 16.04 (6.08) 18.0 (5.16) 13.14 (6.27)

Comprehension Composite raw 32.29 (11.97) 35.52 (6.17) 25.61 (9.01)

Comprehension Composite SS 102.40 (12.15) 105.55 (9.75) 95.90 (11.50)

Note. PPVT-IV reported in standard scores; *raw scores reported; Level 4 of GRADE does not include the Word Recognition subtest. Standard deviations are reported in parentheses.

Writing

Two writing samples were collected per participant upon presentation of one expository and one

narrative writing prompt (see Appendix B) during a single writing session. Two pieces of paper were

provided for each writing sample to be completed. Participants used paper and pencil and were allowed

15 minutes per composition. In addition to reading the prompts, the evaluator wrote the prompts on the

board for the students to view. The elicitation procedure was similar to that of the Florida Writing

Assessment Program, Florida Writes, and progress monitoring program, Writes upon Request Program

(FLDOE, 2009), so that a standard elicitation method was employed similar to other writing

assessments that take place within the context of normal education practices for students in grades one

through four. The writing scale designed for this study, consisting of nine items for microstructure and

three for macrostructure, had good internal consistency, with a Cronbach alpha coefficient of .80.

21

Procedure

Following the attainment of informed consent, students were assessed during the school day on

a group or classroom-wide basis, at a time deemed appropriate by the teachers, to minimize

interruptions to instruction. All participants completed the appropriate level of the GRADE assessment

and one narrative and one expository writing sample. The GRADE required that the listening

comprehension subtest be administered first. The GRADE assessment lasted 45-60 minutes; writing

samples lasted 15 minutes each. Testing was conducted by the primary investigator and trained

research assistants. Data collection took place during the 2008-2009 school year.

Data Analysis

Dependent Measures

Multiple dependent measures were analyzed from the writing samples to describe the elements

of microstructure and macrostructure elements present in the samples. The primary investigator

transcribed the writing samples into a computer database according to Systematic Analysis of

Language Transcript conventions (SALT, Version 8; Miller & Chapman, 2005). The unit of

segmentation was the T-unit, as suggested by Nelson et al. (2004) and consistent with previous

investigations (Nelson & Van Meter, 2007; Puranik et al., 2007, 2008; Scott & Windsor, 2000). Items

that were not part of the sample text, such as “The end”, “Sincerely,”, “That’s all” were not counted

within the dependent measures. After practicing and establishing coding guidelines, the dependent

measures related to microstructure variables were scored following the SALT coding protocol in

Appendix C. Two trained graduate student assistants assisted with the reliability coding. One rater was

primarily responsible for coding of the microstructure variables and a second rater for scoring for

macrostructure variables using the rubric.

Microstructure: Productivity

The microstructure productivity measures values for total number of words (TNW), and T-units

(TNT) were calculated automatically in SALT. TNW is a very frequently used measure of productivity,

and consists of the number of words produced in a given writing sample (Berman & Verhoeven, 2002;

Mackie & Dockrell, 2004; Nelson & Van Meter, 2007; Puranik, 2006; Scott & Windsor, 2000). As with

TNW, TNT is also widely used, and is calculated by SALT as the number of utterances in the transcript

(because the transcript is broken down at the level of the T-unit).

Microstructure: Grammatical Complexity

Numerous measures exist for examining the grammatical complexity of the written product.

Mean length of T-unit (MLTu) is a commonly employed measure that is automatically calculated in

SALT (Berman &Verhoeven, 2002; Nelson & Van Meter, 2007; Puranik, Lombardino, & Altmann, 2007,

22

2008; Scott &Windsor, 2000) by dividing the total number of words by the total number of T-units in a

sample. Additionally, multiple writing variables have been examined in writing research using the unit of

the clause. A clause consists of a group of related words including a subject and a verb (Puranik,

2006). In this study, the total number of clauses (TNC) was calculated in order to compute clause

density (CLD). Both measures have been used in previous examinations of the written product

(Puranik, Lombardino, & Altmann, 2007, 2008; Scott & Windsor, 2000). Clause density (CLD) (Scott &

Stokes, 1995; Scott & Windsor, 2000; Puranik, et al 2007, 2008) was calculated by dividing the total

number of clauses (main and subordinate) in the sample by the total number of T-units across the

sample. In addition, the number of clauses per sentence (CPS) was measured to capture grammatical

complexity at the sentence level. Two sets of sample files were created in SALT to calculate clauses

per T-unit separately from clauses per sentence.

Transcripts within the SALT program were coded for sentence type (complex vs. simple, correct

vs. incorrect), and presence of grammatical errors. A simple sentence consisted of one main clause

and only one verb, while a complex sentence included one main clause plus one or more

embedded/subordinate clauses, two main clauses, or one main clause and verb phrase joined by a

coordinating conjunction. Grammatical errors were defined as errors occurring in verb or pronoun

tense, agreement or case, omitted or incorrect inflection, omitted or substituted grammatical elements,

and violated word order. A sentence without any grammatical errors was considered correct, while a

sentence with one or more errors was deemed incorrect. Occasionally, a grammatical error occurred

across sentences, where the individual sentence was grammatically accurate, but the error was a verb

tense change across the other sentences, or across the body of the paper. Therefore, such errors that

occurred across sentences were not counted as sentence level errors and were indicated in the SALT

transcript as GEX. Consistent with previous investigations, the total number of grammatical errors

(TNGE), and percentage of grammatically correct sentences (% GS) were calculated (Mackie &

Dockrell, 2004; Nelson & Van Meter, 2007; Puranik, 2006; Puranik, Lombardino, & Altmann, 2007).

GEX were counted within the TNGE but did not affect % GS as only within-sentence GEs were involved

in that calculation. Considerations were made regarding the potential influence of participant dialect on

calculation of grammatical errors, the results of which are discussed later.

Microstructure: Lexical Diversity

Lexical diversity, which is thought to indicate vocabulary size and control, is most commonly

measured directly through either the number of different words (NDW) in the written text, or the type-

token ratio (TTR; ratio of different word types to overall words); although, indirect measures of lexical

diversity are often evident in holistic rubrics used for writing assessment (e.g., word choice) (Scott,

2009). In recent years, NDW has become the preferred measure of lexical diversity among those

23

researching writing development as it can be automatically calculated in SALT, is sensitive to

developmental changes, (Nelson, Bahr, and Van Meter, 2004; Puranik, 2006), and is considered to be

relatively free of culture and socioeconomic bias (e.g., Zevenbergen, Whitehurst, & Zevenberger,

2003). Previous researchers have suggested that NDW or TTR are most accurately interpreted when

sample size is controlled for (Scott, 2009; Scott & Windsor, 2000). For this reason, an additional related

measure of lexical properties was incorporated in the present investigation that is not confounded by

writing sample size. Lexical density (LXD) was the proportion of content words (e.g., nouns, verbs,

adjectives) to total words (Scott, 2009). By taking a proportion of content words to total words, each

sample was then measured for lexical density on the same scale regardless of overall sample length,

thereby reducing the impact of sample size.

Macrostructure

The dependent measures related to macrostructure variables (organization, genre-specific text

structure, cohesion) were scored according to an analytic scoring system (see Appendix D for

operational definitions and protocol). The operational definitions for examining levels of organization,

text structure, and cohesion were formed based on key features of informal writing inventories used in

previous investigations (Crawford et al, 2004; Moats, Foorman, & Taylor, 2006; Nelson et al, 2004;

Singer & Bashir, 2002). Organization was examined within the introduction, body, and conclusion of the

product. Products also were examined for use of an appropriate text structure (genre-specific), and

overall cohesion. Each item received a score ranging 1-4. The individual trait scores were combined for

an overall macrostructure composite score. In summary, there were 13 writing variables (9 for

microstructure, 4 for macrostructure) examined, as depicted in Table 3 below:

24

Table 3: Dependent Writing Variables

Level Dependent Measure

Microstructure

Productivity Total number of words (TNW)

Total number of T-units (TNT)

Grammatical Complexity Mean Length T-unit (MLTu)

Total clauses (per sentence) (CPS)

Clause density (# clauses per T-unit) (CLD)

Percentage of grammatical sentences (%GE)

Total grammatical errors (TGE)

Lexical Diversity Number of different words (NDW)

Lexical density (LXD)

Macrostructure

Organization Trait score (1-4)

Text Structure Trait score (1-4)

Cohesion Trait score (1-4)

Macrostructure Composite Combination of the above 3 trait scores

Research Assistant Training

Three graduate students enrolled in the Department of Communication Sciences and Disorders

were recruited as research assistants. Research assistants met specific requirements to participate in

research activities, including 1) use of English as primary language, 2) prior experience working with

school-age populations, 3) completion of 5 hours training before participating in data analysis, and 4)

availability to assist with data analysis during the spring and summer semesters.

Three training sessions were conducted. The first training session served 3 purposes: 1) the PI

provided an overview of the stated problem being examined and the purpose of the investigation; 2) the

PI and research assistant discussed their respective roles and responsibilities, and; 3) the PI and

research assistant discussed scheduling and the assistant’s availability for the semester. The second

training session consisted of an introduction to the SALT program. Each assistant received a data

coding training manual. The PI provided an overview of the manual and the SALT program, and

explained the procedure for assistants to follow. The training manual included an overview of the

problem being addressed & the research questions, overview of how the samples were transcribed into

25

SALT by the PI, scoring protocol for SALT coding of microstructure & macrostructure elements, anchor

samples for coding practice, procedure for establishing reliability with the PI for coding, and proposed

schedule for completion of data transcription and coding. The third session involved guided practice

coding writing samples into the SALT program. For the purpose of training, practice samples that were

independently coded by the PI and reliability coder were expected to reach an agreement criterion of

90% agreement prior to moving on to coding the reliability set. Additional training sessions were

conducted as needed.

Inter-Rater Reliability

Reliability of the dependent measures was established using a randomly-selected sub-sample of

writing samples equaling 25% of the total number of samples collected specifically. Samples selected

for reliability coding consisted of approximately 25% of the samples produced within each genre and

within each grade level. Percent agreement and Cohen's Kappa coefficients were calculated for the

following variable characteristics that required coding in SALT to produce the scores for each of the

dependent variables: T-unit segmentation, clauses per T-unit, clauses per sentence, sentence codes to

indicate grammatical complexity (simple vs. complex) and accuracy (correct vs. incorrect) of the

sentence structure, identification of content words, and identification of grammatical errors. The actual

values for the following variables were automatically calculated in SALT, and therefore did not require

reliability estimates: total words, total T-units, MLTU, total clauses per sentence, clause density,

percentage of grammatical sentences, total grammatical errors, and number of different words.

Percent agreement was calculated by dividing the number of agreements by the number of

disagreements plus agreements and multiplying this score by 100 for each measure. Percent

agreement ranged from 83% to 98% for the microstructure variables, and from 84% to 93% for

macrostructure variables. Percent agreement is a commonly reported measure of inter-observer

agreement; however, a specific disadvantage is that it does not account for the possibility of chance

agreements. Therefore, Cohen's Kappa coefficients also were calculated, by considering the

proportions of observed and chance agreement. Kappa coefficients of >.6 were required to establish

adequate reliability. Kappa values may be interpreted as follows: 0.41-0.60 fair, 0.61-0.80 good, and

over 0.80 very good reliability among raters (Warner, 2008). Kappas ranged from 0.80 to 0.98 for

microstructure variables and from .72 to .90 for macrostructure variables. Therefore, suitable reliability

was established for all coded dependent measures, and therefore judged to be adequate for all

subsequent analyses. The kappa coefficients and percent agreement levels are illustrated in Table 4.

26

Table 4

Cohen's Kappa coefficients and percent agreement.

Coded Variable Kappa statistic* Percent agreement Microstructure 1. T-unit segmentation .95 96% 2. Clause density .80 83% 3. Clauses per sentence .98 98% 4. Total Content words .95 96% 5. Total grammatical errors .91 94% 6. Sentence codes .78 87% Macrostructure 1. Organization .81 84% 2. Text Structure .90 93% 3. Cohesion .72 84% Note. Kappa statistic* = All reported Kappa coefficients were significant at p <.001.

Research Design

This investigation employed multivariate descriptive quantitative research methods to answer

the proposed research questions. In general, the overarching goals of quantitative methods are to test

theories and hypotheses, identify correlational and/or causal relations, and determine group differences

or patterns (Kazdin, 2003). More specifically, descriptive quantitative methods serve to identify

characteristics of observed phenomena, and explore possible correlations among phenomena without

changing them (Leedy & Ormond, 2005). This investigation employed a cross-sectional quantitative

design in that it considered development across grade levels. With multiple dependent variables and

grade level serving as a between-subjects factor, and the z scores of the GRADE Comprehension

Composite as a covariate, two separate Multiple Analyses of Covariances (MANCOVA) were

conducted; one MANCOVA was conducted per genre. To compare microstructure and macrostructure

performance across genres, repeated measures analysis of variance (RM-ANOVA) methods were

utilized. Collectively these analyses were designed to answer the first and second research questions.

To answer the third research question, multiple correlations were conducted. Additionally, preliminary

analyses involving exploratory factor analysis techniques for data reduction were employed.

Power Analysis

Data were analyzed using the Predictive Analytics Software for Windows (PASW), version 17.0

(SPSS, 2009; formerly known as the Statistical Package for Social Science, SPSS). To answer the first

27

and second research questions, data for the dependent writing measures were analyzed using

multivariate techniques (MANOVA) with grade level as a between-subjects factor, and genre as a

within-subjects factor. Adjustments were made for multiple pairwise comparisons to compare

performance of participants in the 3 grades. Correction for Type I error occurred via Bonferroni

correction. Measures of effect size were reported using partial eta squared (using an alpha level of .05,

and ability to detect a large effect size with power equal to or exceeding .70). The maximum number of

variables included in a single MANOVA was five. As such, in order to detect a large effect for a

MANOVA with 5 variables, 3 grade-level groups, α = .05, power = .70, 25 participants per group was

recommended (Stevens, 1997). Consequently, a total of 75-100 participants were required to achieve

sufficient power to conduct the MANOVAs.

28

CHAPTER THREE

RESULTS

The analyses of data and results are presented in five sections. The first section details the

preliminary analyses that were performed to survey the data, check assumptions for the planned

analyses, and utilize data reduction techniques (e.g., exploratory factor analysis). The second section

presents the data on the participants’ performance on the dependent writing measures. Results of two

MANCOVAs conducted to address the first and second research aims to determine the progression of

microstructure and macrostructure elements in the narrative and expository writing of children in

second, third, and fourth grades are reported. The third section includes the results of the correlational

analyses to determine the relations among measures of microstructure and macrostructure, addressing

the third research aim. Findings related to certain demographic characteristics (e.g., ethnicity, SES,

gender) of the sample are explored in the fourth results section. Finally, a post hoc analysis is

presented.

Preliminary Analyses

Data were surveyed for normality and outliers by grade. To detect outliers in the data, all values

were converted to z scores. Outliers were identified using a criterion of the mean plus or minus two

standard deviations. This method revealed a total of 41 univariate outliers across both genres sampled:

total words (3 outliers; 2 narrative, 1 expository), lexical density (2 outliers; 1 narrative, 1 expository),

total T-units (3 narrative outliers), clause density (4 outliers; 2 narrative, 2 expository), clauses per

sentence (4 outliers; 2 narrative, 2 expository), MLTu (5 outliers; 4 narrative, 1 expository), percent

grammatical sentences (7 outliers; 4 narrative, 3 expository), total grammatical errors (3 outliers; 3

narrative), number of different words (1 narrative outlier), organization (2 expository outliers), text

structure (4 outliers; 1 narrative, 3 expository), and cohesion (3 outliers; 1 narrative, 2 expository). Of

the univariate outlier values identified, 41% were in second grade (17), 32% in third grade (13), and

27% in fourth grade (11). Two percent of the cases (41 of 2040 cases) were identified as outliers, an

acceptable proportion within the guideline of >5% outliers identified by the criterion of 2 standard

deviations plus or minus the mean (Field, 2005). The proportion of outliers identified at each grade level

was deemed acceptable as well (approximately 2% per grade level). For outlier scores obtained on

microstructure variables, the scores were substituted with a value equal to the mean plus or minus two

standard deviations, depending on whether the outlier was a low or high outlying value. Outlier scores

from macrostructure variables were not adjusted due to the fact that there was a floor effect for at least

one grade level per macrostructure variable (particularly for grade 2). No multivariate outliers were

detected.

29

Skewness and kurtosis values were reviewed for each variable by grade. Because the variables

were originally measured in different units, skewness and kurtosis values were transformed to z-scores

by dividing the skewness or kurtosis value by its standard error. These z score values for skewness

and kurtosis were compared against values expected by chance alone for a normal distribution. For a

value to be considered of significant concern, a criterion of an absolute value greater than 2.58 was

applied per Field’s (2005) recommendation for small samples.

In second grade, positive skewness was indicated for the following variables: total number of

narrative words (2.96), expository clauses per sentence (4.02), MLTu narrative (3.04) and expository

(2.76), expository organization (3.0), narrative and expository text structure (3.0 and 3.3 respectively),

and expository cohesion (6.4). Narrative percent grammatical sentences exhibited a negative skewness

(-3.39). Second grade variables exhibiting positive kurtosis included: total narrative T units (2.82),

expository clauses per sentence (3.59), expository organization (3.0), and expository cohesion (11.64).

Examination of the data indicated that second graders as a group had scores on a few variables that

clustered more on the low end of the scale, suggesting a possible floor effect, one negatively skewed

variable with scores clustered along the higher end of the scale, suggesting a possible ceiling effect,

and a platykurtic shape for four variables, suggesting a wider range of low values were obtained with

those measures.

In third grade, negative skewness was indicated for narrative and expository percent

grammatical sentences (-3.50 and -3.48 respectively). Positive kurtosis was indicated for narrative

percent grammatical sentences (3.44).

In fourth grade, narrative percent grammatical sentences exhibited a negative skewness

(-3.36) and a positive kurtosis (4.25). Examination of the data indicated that fourth graders’ scores on

this variable clustered near the higher end of the scale and reflected a limited range of scores obtained,

suggesting a possible ceiling effect with less variability for this measure.

Variables with positive skewness were subjected to square root transformation, whereas

variables with negative skewness were first reflected and then subjected to natural log transformations.

Reflection occurred by subtracting the score from the highest achieved score plus one. Variables with

both positive skewness and kurtosis underwent natural log transformations. Transformations improved

the distributions but did not alter the pattern of correlations among variables, nor the pattern of results

reported later. Therefore, analyses were performed with untransformed variables.

Missing data occurred occasionally when a few participants in the sample completed only one of

the two writing samples. Although multiple methods exist for replacing missing values, it was deemed

inappropriate for this investigation as the missing values represented entire writing samples that were

not produced. By grade level, 4 second graders, 1 third grader, and 2 fourth graders completed only

30

one of the two writing samples elicited for this study. There was a concern that those who did not

complete both writing samples may have represented a biased group with lower achievement levels.

This concern was alleviated, however, when comparison of grade level means for the GRADE

measures indicated no significant differences between participants who did or did not complete both

writing samples.

Prior to conducting the main analyses to answer the proposed research questions, additional

analyses were completed to further address assumptions and utilize data reduction techniques before

performing the MANCOVAs.

Assumptions for MANCOVA

Conformity with four assumptions was considered for multiple analysis of covariance

(MANCOVA) due to the potential effects on Type I error rate and power: independent observations,

multivariate normal distribution in each group, equal covariance matrices for all dependent variables,

and homogeneity of regression slopes (Fields, 2005; Stevens 1997). A preliminary ANCOVA supported

the homogeneity of regression slopes assumption for ANCOVA. The observations gathered in this

study were believed to be independent (uncorrelated) because scores of participants in each grade

level were considered independent of scores obtained from participants in the other grade levels.

Multivariate normality could not be checked in PASW, so the assumption of univariate normality was

checked for each dependent variable using the Shapiro-Wilk test (Stevens, 1997). To determine

whether the assumption of homogeneity of covariance matrices was met, the univariate tests of equality

of variances between groups was checked using Levene’s test for each of the dependent variables

(Fields, 2005). Due to unequal group sizes, the homogeneity of the variance-covariance matrices was

checked using Box’s test.

Exploratory Factor Analyses for Microstructure Variables

Two exploratory factor analyses (EFA) were conducted for the purposes of data reduction for

the microstructure and macrostructure variables. Results are first reported for an EFA conducted with

the nine microstructure variables. Second, the results of an EFA for the three macrostructure variables

are reported. The resulting factor scores were saved in the dataset for later use in the MANCOVAs to

address the first and second research questions.

Exploratory Factor Analysis for Microstructure Variables

The nine dependent variables of microstructure, as seen in Tables 3 and 9, were selected

initially based on review of previous research to examine written microstructure. Hence, two exploratory

factor analyses were conducted (one EFA each for narrative and expository genres) to determine what

underlying factors, or components, were represented by the microstructure variables. This was done in

part to confirm the a priori association of variables with certain factors (e.g., such as total number of

31

words and total number of T units being associated with the factor of productivity), and also as a data

reduction technique prior to conducting MANCOVAs. The analyses were conducted in PASW using a

principal component analysis factor extraction method. Three criteria were used to determine the

number of factors to rotate: the a priori hypothesis that the measure of microstructure consisted of at

least 3 dimensions, the scree test, and the interpretability of the factor solution.

Based on the scree plots and presence of eigenvalues over 1.0, four factors were rotated using

direct oblimin (Delta = 0), an oblique rotation technique. An oblique rotation technique was selected,

opposed to an orthogonal rotation technique, due to the theoretically-based expectation that some of

the nine microstructure variables could be correlated. Examination of a bivariate correlation matrix

confirmed that each of the nine grammatical complexity variables were indeed correlated with 1 or more

other variables, but not to the extent that multicollinearity would be of concern.

The rotated solution yielded a four-factor solution. The results are reported in Tables 7 and 8. In

the narrative genre, the first factor (productivity) accounted for 34.31% of the variance, the second

factor (grammatical complexity) 25.39%, the third factor (grammatical accuracy) 15.3%, and the fourth

factor (lexical diversity) 11.60%, totaling 86.6% of the variance. Similarly in the expository genre, the

first factor (productivity) accounted for 37.56% of the variance, the second factor (grammatical

complexity) 25.27%, the third factor (grammatical accuracy) 13.79%, the fourth factor (lexical diversity)

9.99%, totaling 86.61% of the variance. For both genres, three variables loaded on the first factor of

productivity (total words, total T-units, NDW), three variables loaded onto the second factor of

grammatical complexity (clause density, clauses per sentence, MLTu), two variables loaded onto the

third factor of grammatical accuracy (percent grammatical sentences, total grammatical errors), and

one variable loaded onto the fourth factor of lexical diversity (lexical density).

These findings identified two grammatical factors (factor 2, grammatical complexity; factor 3,

grammatical accuracy). For both genres, 3 variables loaded on the first factor (clause density, clauses

per sentence, and MLTU), whereas 2 others (% grammatical sentences, # grammatical errors) loaded

on the second factor. The factor loadings illustrated in Table 6 below did not completely align with

predictions because it was anticipated that the proposed grammatical complexity measure was

representative of a single level of microstructure measured by five variables. However, based on the

results of the EFAs, it appears that this measure reflected two different, but related, grammatical

dimensions of microstructure. In comparison to previous research, the second factor in the present

analysis resembles the accuracy factor (including variables of syntactic errors per T-unit and %

grammatically correct sentences) identified by the factor analysis reported by Puranik et al (2008). As

such, it was determined that the two grammatical variables loading onto the second factor in the

32

present study could actually serve as a fourth factor measured within the microstructure measure.

Therefore, this fourth factor level was named grammatical accuracy.

The results also indicated that NDW loaded onto the first factor with total words and total T-units

as measures of productivity. NDW was initially proposed as a measure of lexical diversity. Based on

these results, the microstructure measure consisted of four factors overall: productivity, grammatical

complexity, grammatical accuracy, and lexical diversity.

Exploratory Factor Analysis for Macrostructure Variables

The dependent variables for the macrostructure measure (organization, text structure, and

cohesion) were selected initially based on review of previous research to examine the macrostructure

of written language. EFA was again utilized as a data reduction technique and to determine the

dimensionality of these three variables. Two exploratory factor analyses were conducted (one EFA

each for narrative and expository macrostructure variables). Analysis was conducted in PASW using a

principal component analysis factor extraction method. Three criteria were used to determine the

number of factors to rotate: the a priori hypothesis that the measure of macrostructure (utilizing three

variables) was unidimensional, the scree test, and the interpretability of the factor solution. Based on

the scree plots and presence of eigenvalues over 1.0, only one factor was extracted. Therefore, factor

rotation could not be conducted. Examination of a bivariate correlation matrix confirmed that each of

the three macrostructure variables were strongly inter-correlated. Moreover, examination of an R-matrix

computed as part of the EFA revealed R values ranging from .70 to .77 in the narrative genre, and from

.62 to .68 in the expository genre. However, with none greater than 0.90, multicollinearity was of little

concern.

Tables 7 and 8 illustrate the eigenvalues and standardized loadings for the macrostructure

variables and the single extracted factor for each genre. For both genres, the analysis yielded one

interpretable factor for macrostructure. The macrostructure factor accounted for 81.64% and 77.22% of

the overall variance in the narrative and expository genres, respectively. For both genres, all three

macrostructure variables loaded on the identified macrostructure factor (organization, text structure,

and cohesion). The factor loadings illustrated in Table 8 below aligned with predictions that the

macrostructure measure was in fact unidimensional and yielded one factor measuring macrostructure

with three variables.

Multiple Analysis of Covariance

Because the grade level groups were not all equivalent on their reading scores (as measured by

the GRADE Comprehension Composite scores), there was a possibility of preexisting differences on

reading ability. Considering the potential impact of reading ability on writing performance, this was an

important factor to explore (Fitzgerald & Shanahan, 2000). To assess this, the GRADE Comprehension

33

Composite scores, after being converted to z scores, were used as a covariate for the MANOVAs. A

preliminary analysis to evaluate the homogeneity of slopes assumption indicated that the relationship

between the covariate (z score for GRADE Comprehension Composite), and the dependent variables

did not differ significantly as a function of the independent variable (grade level) in either the narrative

(F (2, 80) = .55 to 1.79, p = .17 to .58, η2 = .01 to .04), or expository genre (F (2, 77) = .13 to 2.704, p =

.07 to .88, η2 = .003 to .06).

Following a review of the preliminary analyses, two MANCOVAs were conducted to measure

the effects of grade level within each genre on the various microstructure and macrostructure variables,

controlling for reading comprehension scores. The regression-based factor scores for the factors

identified by exploratory factor analyses were saved as variables and utilized in the MANCOVAs. Table

9 shows the final factors of analysis with the respective dependent variables that were analyzed within

each factor. The factor scores of the following five factors were examined in each genre for differences

between grade levels: Productivity, Grammatical Complexity, Grammatical Accuracy, Lexical Diversity,

and Macrostructure. Within the factor of productivity, there were three variables (total words, total T-

units, number of different words), factor of grammatical complexity had 3 variables (clause density,

clauses per sentence, and MLTU), grammatical accuracy had 2 variables (percent grammatical

sentences, total grammatical errors), lexical diversity was represented by one variable (lexical density),

and macrostructure factor represented by three variables (organization, text structure, and cohesion).

Tables 10 and 11 contain the grade level means and standard deviation for the dependent

measures specifically. Tables 12 and 13 contain the adjusted means (accounting for the effects of the

covariate) and standard deviations for the dependent factors across grades and genres. The

performance of the three grades was compared through pairwise comparisons, adjusted for multiple

comparisons and corrected for Type I error rate via a Bonferroni correction. The effect size index

reported is partial eta squared (η2) to indicate the magnitude of the association between the effect and

the dependent variable. Partial eta squared (η2) effect sizes may have values ranging from 0 to 1.0, and

can be interpreted as <.10 = small, .10-.25 = medium, >.25 = large (Warner, 2008).

Two separate, one-way multiple analyses of covariances (MANCOVA) were conducted to

determine the effect of grade in both genres (narrative, expository) on the identified factor scores for the

5 factors of productivity, grammatical complexity, grammatical accuracy, lexical diversity, and

macrostructure. The Box M test (using α =.01 as the criterion for significance) did not indicate a

significant violation of the assumption of homogeneity of variance/covariance matrices across groups.

Inter-correlations between measures ranged from .01 to .52 for the narrative genre and -.08 to .40 for

the expository genre. None of the correlations among outcome variables was sufficiently large to raise

concerns about multicollinearity. Pillai’s trace was selected as the multivariate test statistic due to

34

unequal group sizes (Field, 2005). Pairwise comparisons were conducted to compare performance

across grades, utilizing an adjustment for multiple comparisons and corrected for Type I error rate using

a Bonferroni correction. Therefore, the significance level was established as p <.003 (p < .05 / 3 levels

of grade multiplied by 5 dependent variables).

When the GRADE Comprehension Composite scores were not statistically controlled, grade

level effects were evident for some of the dependent variables in each genre. The main effect for grade

level in the final MANCOVA using the GRADE Comprehension Composite z-scores as a covariate was

also statistically significant and yielded nearly parallel results in terms of grade level effects on the

dependent writing variables. A significant advantage to adding a covariate to the analysis was that it

reduced the unexplained variance in the dependent variables, reducing the error term in the model.

The MANCOVAs (one for each genre) to examine the effects of grade level, controlling for

reading comprehension, revealed a significant multivariate effect of grade for both the narrative (Pillai’s

trace = .62, F(10,158) = 6.572, p < .001, partial η2 = .29), and expository (Pillai’s trace = .50, F(10,152)

= 5.328, p < .001, partial η2 = .26), genres, with large effect sizes. By analyzing the component

measures, a significant main effect was observed for the narrative productivity factor, F(2, 82) = 29.971,

p < .001, partial η2 =.42, with a large effect size. This main effect was indicated by an overall increase

in productivity at every grade level, as reflected by total words, total T-units, and number of different

words. Pairwise comparisons revealed that children in second grade scored significantly lower on

productivity (M=-.86) relative to the third (M=.00) and fourth (M=.83) grade groups. The third and fourth

grade groups differed significantly from each other in productivity as well, with increased productivity

levels indicated for fourth grade. A significant main effect also was observed for narrative

macrostructure F(2, 82) = 12.335, p <.001, partial η2 = .31, with a large effect size. Second grade

scored significantly lower (M=-.81) than both third (M=.31) and fourth (M=.41) grades; however, third

grade did not differ significantly from fourth. In the narrative genre, there were no significant differences

among the 3 grade level groups in scores on grammatical complexity, grammatical accuracy, and

lexical diversity.

Similar results were obtained in the expository genre with a significant main effect for

productivity, F(2, 79) = 28.162, p < .001, partial η2 =.42, with a large effect size. Pairwise comparisons

indicated that as with the results for narrative productivity, second grade scored significantly lower on

expository productivity (M=-.85) than third (M=.04) and fourth grades (M=.81) The third and fourth

grade groups differed significantly from each other in expository productivity as well, with greater

productivity in the fourth grade. Additionally, there was a significant main effect for expository

macrostructure, F(2, 79) = 4.621, p <.01, partial η2 = .12, with a medium effect size. Second grade (M=-

.43) scored significantly lower than both third (M=.05) and fourth grade (M=.38). Even though there was

35

a trend for fourth graders to score higher on macrostructure, there was not a significant difference

between third and fourth grades. Furthermore, a significant main effect was detected for grammatical

complexity, F(2, 79) = 4.487, p <.01, partial η2 = .10, with a medium effect size. Surprisingly, second

grade (M= .48) scored significantly higher than third (M=-.20) and fourth grade (M=-.24). Third and

fourth grades were not significantly different from each other. There were no significant differences

among the 3 grade level groups in scores on grammatical accuracy and lexical diversity in expository

samples.

Differences between Genres

To determine whether differences on microstructure and macrostructure performance existed

between genres, a series of 5 repeated measures ANOVAs were conducted (one per factor DV). The

factor scores for each dependent variable in the narrative and the expository genre were first compared

across all 3 grade levels. Correlations between narrative and expository measures across and within

grade levels are shown in Table 16. Additionally, five profile plots illustrate these comparisons for each

variable in Figures 1 through 5. For overall productivity performance, there was no significant difference

across genres; the narrative and expository productivity scores were very strongly correlated (r = .73,

p<.001). Within grade level, narrative and expository productivity values were highly correlated within

second (r =.66, p<.001) and fourth grades (r =.66, p<.001). Third grade demonstrated a moderately

sized but statistically insignificant correlation (r =.34, p =.06). In Figure 1, the profile plot illustrates how

in both genres, productivity gradually increased with each subsequent grade level at similar rates.

Regarding grammatical complexity, there was no overall significant difference across genres;

narrative and expository grammatical complexity scores were negatively correlated (r = -.31, p<.01).

Within grade level, narrative and expository values appeared to differ for second grade (narrative M = -

.05; expository M = .45), but did not reach statistical significance. Values in second grade were highly

negatively correlated (r =-.53, p<.01). Also in examining the results for fourth grade, a trend toward

greater values in the narrative genre is visible (narrative M = .05; expository M = -.33). However, this

difference was not statistically significant. The profile plot in Figure 2 illustrates that grammatical

complexity remained nearly stable across subsequent grade levels in the narrative genre, but

demonstrates a sharp decrease in the expository genre between second and third grades, with similar

scores between third and fourth grades.

No significant differences in grammatical accuracy between genres were detected across

grades; these scores were negatively correlated overall, r =-.31, p=.005. Within grade levels, a

significant correlation was detected within second grade (r =.44, p<.05). The profile plot below in Figure

3 shows that for both genres, grammatical accuracy levels were similar between second and third

grade, but decreased sharply between third and fourth.

36

No significant differences in lexical diversity were noted between genres among grade levels.

These scores were insignificantly correlated, (r =-.08, p=.48). It is evident from the profile plot in Figure

4 that lexical diversity levels in the third grade were almost equivalent across genres. No significant

correlations were detected within grade levels. There is a visible trend for increased lexical diversity

between third and fourth grades in the narrative genre. But in the expository genre, visual analysis

suggests second grade performed slightly better than third and fourth grades, but third and fourth

grades were very similar to each other.

For macrostructure performance, no differences were noted between genres; scores from each

genre were not significantly correlated (r =.18, p =.12). In the profile plot below (Figure 5), a sharp

increase is seen in the narrative genre from second to third grade; however, values are similar for third

and fourth grades. In the expository genre, a more gradual upward trend is visible between subsequent

grade levels.

Productivity by Grade Level

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

Prod

uctiv

ity M

ean

Fact

or S

core

NarrativeExpository

2 3 Grade Level

4

Figure 1: Productivity; Narrative and Expository

37

Grammatical Complexity by Grade Level

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

Gra

mm

atic

al C

ompl

exity

Mea

n Fa

ctor

Sco

res

NarrativeExpository

2 3 4 Grade Level

Figure 2: Grammatical Complexity: Narrative and Expository

-0.3

-0.25-0.2

-0.15-0.1

-0.050

0.050.1

0.150.2

0.25

Gra

mm

atic

al A

ccur

acy

Mea

n Fa

ctor

S

core

s NarrativeExpository

Grammatical Accuracy by Grade Level

2 3 4

Grade Level Figure 3. Grammatical Accuracy; Narrative and Expository

38

Lexical Diversity by Grade Level

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4Le

xica

l Div

ersi

ty M

ean

Fact

or S

core

s

NarrativeExpository

2 3 4

Grade Level Figure 4. Lexical Diversity; Narrative and Expository

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

Mac

rost

ruct

ure

Mea

n Fa

ctor

Sco

res

NarrativeExpository

Macrostructure by Grade Level

2 3 4

Grade Level Figure 5. Macrostructure; Narrative and Expository

Inter-Correlations among Measures of Microstructure and Macrostructure

The third research aim was addressed via correlational analyses to determine the relations

among the participants’ performance on the microstructure and macrostructure variables. Pearson

correlations are reported in the correlation matrices in Table 15 for both the narrative and expository

genres for the entire sample of participants. Grade level correlations among measures are discussed

39

subsequently and reported in Table 16. In the narrative genre, the correlations between the writing

factors indicated that the factors were not highly interrelated, with the exception of productivity and

macrostructure factors. Based on data obtained in the narrative genre, this was the only statistically

significant correlation detected (r = .61, p <.001). A similar correlation between productivity and

macrostructure was detected in the expository genre (r = .50, p <.001). In slight contrast to narrative

results, the data collected in the expository genre also detected small negative correlations between

expository macrostructure and grammatical accuracy (r = -.25, p <.05), and grammatical complexity (r =

-.23, p <.05).

Grade level correlations among dependent variables (factors) confirm in part as well as extend

the correlational analysis from the overall comparisons. In the narrative genre, correlations among

variables in the third and fourth grade groups mirrored the overall results, with only one significant

correlation between productivity and macrostructure (r = .55, p <.001; r = .41, p <.05; 3rd and 4th grades

respectively). Correlations among variables in the second grade group also paralleled the overall

comparison with a significant correlation between productivity and macrostructure (r = .73, p <.001).

However, a second significant correlation was detected within second grade between grammatical

complexity and macrostructure (r = .41, p <.05).

In the expository genre, the correlation between productivity and macrostructure were found

within both second (r = .56, p <.001), and third grade groups (r = .54, p <.001). However, within second

grade, a negative correlation was detected between grammatical accuracy and lexical diversity (r = -

.44, p <.05). The only correlation detected within the fourth grade group was between productivity and

grammatical complexity (r = .40, p <.05).

Findings Related to Ethnicity, Gender, and SES Differences

Although the primary goal of this investigation was to ascertain whether differences could be

detected on multiple dependent variables for writing based on grade level and genre, it also was

important to consider whether the measures used were sensitive to additional factors such as

differences in gender, ethnicity and/or socioeconomic status. Previous research has suggested that

gender may affect levels of written productivity. For example, in a post hoc analysis, Nelson and Van

Meter (2007) reported the relative volubility of female participants, in grades one through five, on

measures of productivity similar to those utilized in the present investigation (e.g., total words, total T-

units, NDW). Concerns also have been noted in interpretations of writing analyses in previous

investigations for children who speak a dialect other than Standard American English (SAE) (Ball, 2006,

Moats, Foorman, & Taylor, 2006; Terry, 2006). Specifically, the main concern was whether dialectal

differences could inflate the grammatical error rate for such students. Moreover, SES has been

40

indicated as an influential factor, in that children from low SES families face an increased risk for

deficient language and literacy development and academic difficulty (Wasik & Hendrickson, 2004).

In both the narrative and expository genres, MANCOVAs showed the previously revealed main

effect for grade, but no main effect for ethnicity, gender, or SES. Descriptive statistics, viewed in Table

17, were reviewed, and no noticeable trends within the data were detected. These results are further

examined in the discussion.

Post Hoc Analysis

Inspection of the samples suggested that children may not have written a sample consistent

with the prompt's intent. To help address the issue of whether the writing samples truly reflected a

narrative versus an expository genre text structure, four additional coders were recruited to review a

subset of writing samples and identify which genre (narrative vs. expository) the writing sample most

closely resembled based on given definitions of each. (Narrative was defined as telling a story, often

about personal events or other life experiences, it may be fictional or non-fictional, and can include

novels, personal stories or letters, and short stories. Expository was defined as conveying facts,

describing procedures, explaining something, sharing basic information, relating cause-effect, compare-

contrast relationships, and/or arguing a point of view, and may include term papers, procedural

documents, manuals, essays, editorials, letters to the editor. The category of "not sure" was defined as

a genre structure that could not be determined from the given definitions for narrative and expository).

The coders were blind to the conditions of the study and had no prior exposure to or knowledge of the

writing prompts utilized or the writing samples collected.

A randomly selected set of writing samples was utilized for this task, approximately 25% of the

entire sample, with balanced representation from each grade and genre level. The results indicated that

in 71% of cases at least 3 of the 4 raters coded the same genre. They most reliably coded the

expository genre (k = 0.48), followed closely by the narrative genre (k = 0.40). Coders also noted a few

instances in which they could not determine whether a particular sample reflected either narrative or

expository genre, and reported noticing features of each in such samples (suggesting a hybrid genre,

as noted in Scott, 2009). This happened approximately 4 times per rater, or for approximately 10% of

the rated sample they rated.

41

Table 5

Four-factor solution of written microstructure.

Factor Initial Eigenvalue % of Variance Rotations Sums of Squared Loadings Narrative 1 3.088 34.309 3.062 2 2.285 25.392 2.241 3 1.377 15.299 1.400 4 1.044 11.602 1.264 5 0.489 5.436 -- 6 0.382 4.245 -- 7 0.235 2.616 -- 8 0.056 0.619 -- 9 0.043 0.481 -- Expository 1 3.380 37.560 3.074 2 2.274 25.272 2.379 3 1.241 13.794 1.719 4 0.899 9.987 1.340 5 0.627 6.965 -- 6 0.310 3.448 -- 7 0.211 2.347 -- 8 0.037 0.411 -- 9 0.019 0.216 --

42

Table 6

Factor loadings.

Factor Microstructure Variables 1 2 3 4 Narrative 1. Total words .96 -.02 .08 -.05 2. Total T-units .92 -.31 -.02 -.14 3. Lexical density .15 -.33 .01 .90 4. Number of different words .97 -.01 .04 .01 5. Clause density .21 .86 .05 .23 6. Clauses per sentence .16 .77 .33 -.25 7. Mean length of T-unit .11 .83 -.05 .32 8. Percent grammatical sentences .21 -.25 .84 .07 9. Total grammatical errors .47 .05 -.74 -.03 Expository 1. Total words .90 .36 .15 .10 2. Total T-units .75 .63 .05 -.07 3. Lexical density -.39 .24 -.03 .84 4. Number of different words .91 .33 .13 .08 5. Clause density .50 -.74 .23 .13 6. Clauses per sentence .55 -.61 .14 -.10 7. Mean length of T-unit .33 -.65 .28 .34 8. Percent grammatical sentences -.27 .49 .74 .05 9. Total grammatical errors .54 .07 -.71 .20 Note. Factor 1= Productivity; Factor 2 = Grammatical Complexity; Factor 3 = Grammatical Accuracy; Factor 4 = Lexical density; MLTU = mean length of T-unit.

Table 7

One-factor solution of macrostructure.

Factor Initial Eigenvalue % of Variance Rotations Sums of Squared Loadings Narrative 1 2.449 81.644 -- 2 0.324 10.795 -- 3 0.227 7.560 -- Expository 1 2.317 77.217 -- 2 0.379 12.641 -- 3 0.304 10.141 --

43

44

Table 8 Factor loadings.

Factor Macrostructure Variables Macrostructure Narrative 1. Organization .92 2. Text Structure .91 3. Cohesion .88 Expository 1. Organization .87 2. Text Structure .87 3. Cohesion .89

Table 9

Factors and Respective Dependent Variables Analyzed via MANCOVA

Factor Dependent Measure

Productivity Total words

Total T-units

Number of Different Words

Grammatical Complexity Mean Length T-unit

Clauses per sentence

Clause density (# clauses per T-unit)

Grammatical Accuracy Percentage of grammatical sentences

Total grammatical errors

Lexical Diversity Lexical density

Macrostructure Organization Trait Score

Text Structure Trait Score

Cohesion Trait Score

Table 10. Descriptive statistics for dependent measures; narrative genre.

Grade 2 Grade 3 Grade 4 Measure M SD M SD M SD _________________________________________________________________________________________________________ Productivity Total Words 24.27 12.22 53.24 25.57 77.59 34.56 Total T-units 3.46 1.75 6.45 3.28 9.59 4.42 Number of Different Words 18.31 7.05 36.88 14.58 47.33 17.06 Grammatical complexity

Mean Length T-unit 7.51 2.87 8.49 2.39 7.98 2.02 Clauses (per sentence) 1.83 0.77 1.96 0.55 1.74 0.50 Clause density (# clauses per T-unit) 1.40 0.40 1.43 0.34 1.46 0.29 Grammatical accuracy

Percentage of grammatical sentences 0.81 0.29 0.84 0.19 0.82 0.19 Total grammatical errors 0.65 0.79 1.06 1.03 1.92 1.66 Lexical diversity Lexical density 0.52 0.08 0.53 0.05 0.55 0.05 Macrostructure

Organization 4.92 1.29 6.79 1.59 6.78 1.39 Text Structure 1.54 0.86 2.76 0.83 2.52 0.85 Cohesion 1.31 0.47 2.00 0.88 1.93 0.68

45

Table 11. Descriptive statistics for dependent measures; expository genre. Grade 2 Grade 3 Grade 4 Measure M SD M SD M SD _________________________________________________________________________________________________________ Productivity Total Words 27.77 12.00 53.97 22.11 73.46 29.55 Total T-units 3.73 1.22 6.62 2.96 8.84 4.20 Number of Different Words 20.85 7.59 37.56 12.71 47.85 16.92 Grammatical complexity

Mean Length T-unit 7.58 3.05 8.58 2.23 8.33 1.87 Clauses (per sentence) 1.49 0.59 1.83 0.44 2.05 0.58 Clause density (# clauses per T-unit) 1.25 0.32 1.58 0.30 1.61 0.43 Grammatical accuracy

Percentage of grammatical sentences 0.78 0.27 0.85 0.21 0.76 0.25 Total grammatical errors 0.92 0.79 1.75 1.50 2.04 1.78 Lexical diversity Lexical density 0.62 0.09 0.60 0.07 0.61 0.06 Macrostructure

Organization 4.04 1.25 4.66 1.49 4.85 1.40 Text Structure 1.31 0.55 1.81 0.78 1.92 0.79 Cohesion 1.15 0.46 1.34 0.48 1.46 0.58

46

47

Table 12. Adjusted means and standard deviations for dependent measures; narrative genre.

Grade 2 Grade 3 Grade 4 Factor M SD M SD M SD F Ratio Effect _________________________________________________________________________________________________________ Productivitya -.85 .15 .00 .14 .83 .16 29.97** .42 Grammatical complexity -.15 .20 .10 .18 .03 .20 0.47 .01 Grammatical accuracy .14 .20 .08 .18 -.23 .20 1.07 .03 Lexical diversity -.21 .20 -.09 .18 .33 .20 2.04 .05 Macrostructureb -.81 .16 .31 .15 .41 .17 18.07** .31

Note. *p < .01; **p <.001; Values are reported in factor scores; a = all 3 pairwise comparisons among grade levels were significant; b = 2 of 3 pairwise comparisons were significant. Table 13. Adjusted means and standard deviations for dependent measures; expository genre. Grade 2 Grade 3 Grade 4 Factor M SD M SD M SD F Ratio Effect _________________________________________________________________________________________________________ Productivitya -.85 .16 .04 .14 .81 .16 28.16** .42 Grammatical complexityb .48 .19 -.20 .18 -.24 .20 4.62* .11 Grammatical accuracy .18 .91 -.05 .17 -.10 .20 0.66 .02 Lexical diversity .14 .20 -.15 .18 .02 .21 0.57 .01 Macrostructureb -.43 .19 .05 .18 .38 .20 4.49* .10 Note. *p < .01; **p <.001; Values are reported in factor scores; a = all 3 pairwise comparisons among grade levels were significant; b = 2 of 3 pairwise comparisons were significant.

Table 14. Overall and grade level correlations among narrative and expository values for DVs Grade Level Variable 2-4 2 3 4 __________________________________________________________________________________ Productivity .73* .66* .34 .66*

Grammatical complexity -.31^ -.53^ -.22 -.17

Grammatical accuracy .31^ .44+ .17 .25

Lexical diversity -.08 -.25 .01 .31

Macrostructure .18 .23 .16 -.27

Note: * p<.001; ^p<.01; +p<.05

Table 15

Correlation Matrix – Narrative (factor scores) 1. 2. 3. 4. 5. 1. Productivity -- .05 -.02 .08 .61*

2. Grammatical complexity -- -.04 -.16 .12

3. Grammatical accuracy -- -.04 .10

4. Lexical diversity -- .07

5. Macrostructure --

Note: * p<.001; ^p<.01; +p<.05 Correlation Matrix – Expository (factor scores) 1. 2. 3. 4. 5. 1. Productivity -- -.05 -.16 -.10 .50*

2. Grammatical complexity -- .19 .22 .23+

3. Grammatical accuracy -- .14 -.25+

4. Lexical diversity . -- -.06

5. Macrostructure --

Note: * p<.001; ^p<.01; +p<.05

48

Table 16 Second Grade Correlation Matrix – Narrative (factor scores) 1. 2. 3. 4. 5. 1. Productivity -- .16 .33 -.14 .73*

2. Grammatical complexity -- .23 -.17 .41+

3. Grammatical accuracy -- -.06 .19

4. Lexical diversity -- -.09

5. Macrostructure --

Note: * p<.001; ^p<.01; +p<.05 Third Grade Correlation Matrix – Narrative (factor scores) 1. 2. 3. 4. 5. 1. Productivity -- .02 .15 -.02 .55*

2. Grammatical complexity -- -.25 -.07 -.08

3. Grammatical accuracy -- .18 .10

4. Lexical diversity -- .14

5. Macrostructure --

Note: * p<.001; ^p<.01; +p<.05 Fourth Grade Correlation Matrix – Narrative (factor scores) 1. 2. 3. 4. 5. 1. Productivity -- -.05 .02 -.09 .41+

2. Grammatical complexity -- -.16 -.31 -.04

3. Grammatical accuracy -- -.06 .26

4. Lexical diversity -- -.09

5. Macrostructure --

Note: * p<.001; ^p<.01; +p<.05

49

Table 16 - Continued Second Grade Correlation Matrix – Expository (factor scores) 1. 2. 3. 4. 5. 1. Productivity -- -.31 .15 .25 .56*

2. Grammatical complexity -- .27 .18 -.34

3. Grammatical accuracy -- -.44+ -.24

4. Lexical diversity -- -.15

5. Macrostructure --

Note: * p<.001; ^p<.01; +p<.05 Third Grade Correlation Matrix – Expository (factor scores) 1. 2. 3. 4. 5. 1. Productivity -- .18 -.25 -.12 .54*

2. Grammatical complexity -- -.02 .13 -.11

3. Grammatical accuracy -- -.06 -.23

4. Lexical diversity -- .01

5. Macrostructure --

Note: * p<.001; ^p<.01; +p<.05 Fourth Grade Correlation Matrix – Expository (factor scores) 1. 2. 3. 4. 5. 1. Productivity -- .40+ -.01 -.21 .34

2. Grammatical complexity -- .22 .32 -.04

3. Grammatical accuracy -- .01 -.19

4. Lexical diversity -- .07

5. Macrostructure --

Note: * p<.001; ^p<.01; +p<.05

50

Table 17. Descriptive statistics for five writing factors by ethnicity, gender, and SES. Writing Factor Characteristic 1 2 3 4 5 N E N E N E N E N E Gender Male -.26 -.35 .02 -.16 -.09 -.12 .31 .11 -.15 -.03 Female .07 .15 -.17 -.10 .04 -.08 -.18 -.31 .15 -.07 Ethnicity Caucasian .01 .01 -.09 -.12 -.43 -.24 -.22 .03 .08 .00 African American .06 -.04 .15 -.22 -.22 -.42 .10 -.38 -.08 .22 Asian American -.29 -.35 .07 -.36 -.05 .05 .06 .08 .30 -.16 Hispanic -.45 -.23 -.50 .84 .24 .09 .28 .28 -.06 .05 Other -.19 -.12 -.12 -.27 .74 .29 .42 -.21 -.13 -.73 Free/Reduced Lunch Receiving -.13 -.34 -.19 -.28 -.46 -.52 -.22 -.40 -.03 -.19 Not Receiving -.08 .03 .00 -.04 .23 .17 .25 .11 .05 .05 Note. N = Narrative; E = Expository mean factor scores. Free/reduced lunch rate = measure of socioeconomic status (SES). Factor 1 = Productivity; Factor 2 = Grammatical Complexity; Factor 3 = Grammatical Accuracy; Factor 4 = Lexical Diversity; Factor 5 = Macrostructure.

51

CHAPTER 4

DISCUSSION

The purpose of this study was to examine multiple dimensions of written language produced by

children in grades 2, 3, and 4 in a narrative and an expository writing sample. A primary goal of this

investigation was to determine the developmental progression of linguistic elements of written

microstructure and macrostructure demonstrated by typically developing students in three elementary

grades. Four aspects of microstructure were examined: productivity, grammatical complexity,

grammatical accuracy, and lexical diversity. Macrostructure was measured by trait scores for

organization, text structure, and cohesion. A secondary aim of this study was to compare performance

on the multiple dependent variables for writing across two different discourse genres (e.g., narrative,

expository) commonly utilized in the educational setting. Third, the inter-correlations among dependent

variables across and within grade levels were explored. The discussion is presented in three sections,

first addressing the results of the present study, then focusing on limitations and offering suggestions

for future research, and concluding with educational implications.

Effect of Grade Level on Microstructure

The first goal of this investigation was to determine whether there were differences among

grades and between genres in linguistic microstructure elements. It was hypothesized that statistically

significant differences would be detected among grade levels on microstructure measures of

productivity and grammatical complexity in both narrative and expository genres. As hypothesized,

differences were indeed found between grade levels (second-third, second-fourth, and third-fourth) for

productivity in both genres based on MANCOVA results. Participants in third and fourth grades used

more words, produced more T-units, and had greater numbers of different words than students in

second grade, and fourth graders greater than third. These findings are consistent with results from

previous investigations demonstrating that measures of productivity are sensitive to changes in grade

and age levels in more than one genre (Berman & Verhoeven, 2002; Houck & Billingsley, 1989; Nelson

& Van Meter, 2007; Puranik et al, 2008).

Partially confirming the hypothesis for grammatical complexity, differences were found between

two subsequent grade levels (second and third), as well as between second and fourth grades in the

expository genre. However, similar to the findings of Puranik et al. (2008), no significant differences in

grammatical complexity were indicated between third and fourth grade levels. This suggests that levels

of expository grammatical complexity, as measured by MLTU, clauses per sentence, and clause

density may plateau at third grade, at least on writing assignments similar to those used in this study

and Puranik et al.

52

Interestingly, fourth graders obtained lower levels of expository grammatical complexity than

both second and third graders. This finding supports Nelson and Van Meter’s (2007) suggestion that

grammatical complexity may actually decrease with later grades as a natural part of the developmental

progression of writing ability. Nelson and Van Meter noted that the older (fifth grade) students in their

sample produced more simple, correct sentences than did younger students in lower grades. Multiple

possible reasons exist for the finding that the expository writing of fourth grade students was less

grammatically complex than for the younger students. One possibility is that there was a sampling

effect for fourth grade. In other words, the fourth grade sample may have consisted of students who

were obtaining lower levels of achievement than other grade level groups. However, inspection of

fourth grade mean scores on independent measures failed to support this notion. Another possible

explanation is the concept of a "fourth grade slump”, evident when literacy difficulties manifest in later

elementary school (i.e., fourth grade), despite adequate achievement in earlier grades (Borman,

Dowling, & Schenk, 2008; Davis & Compton, 2008; Sanacore and Palumbo, 2009). However, using this

explanation, it is unclear why the fourth grade group would outperform second and third graders on

some writing measures (productivity, macrostructure), suggesting there was not a downward trend

across all writing measures. The trend for fourth graders to outperform second and third graders on

some variables, and not others, suggests a third, and perhaps most plausible explanation, that a

linguistic trade-off was occurring to meet task and genre demands. It may be that fourth graders'

linguistic resources were devoted implicitly to producing increased productivity and macrostructure

levels, at the cost of grammatical complexity. Other authors have noted a linguistic trade-off effect in

relation to writing for students with language-learning disabilities (Singer and Bashir, 2004), and

students with English as a second language (Li, 2000). They posit that the more explicit linguistic

features, such as grammatical accuracy, appear to trade-off in accuracy as a function of task demands

and increased cognitive load. Thus, even though the independent measures indicated the fourth

graders in this study were exhibiting typical development, for tasks involving higher cognitive load, they

may have been focusing on the more explicit features of their text production, such as productivity (total

words), and macrostructure (organization & text structure) at the expense of grammatical accuracy

(grammatical errors) or grammatical complexity (complex sentence structures). The topic of “trade-off”

between linguistic features is revisited later to help explain additional findings of the present

investigation.

In contrast to expectations, no grade differences were found for grammatical complexity in the

narrative genre. This result is dissimilar from the results of Nelson and Van Meter (2007), who found a

significant difference on MLTu between second and third grades in a narrative genre. The grade level

means and standard deviations for MLTu in the present investigation were very similar to the sample

53

examined by Nelson and Van Meter, so it is initially puzzling why significant grade level differences

were not found. However, examination of grade level means in Table 12 for clause density and clauses

per sentence clearly indicate nearly the same scores across the three grade levels in the present

investigation. Therefore, it may be that clauses per sentence and clause density were not sensitive

enough to detect differences between subsequent grade levels in grammatical complexity in a narrative

genre. These values for clauses per sentence and clause density may have masked the effects of

MLTu when these variables were combined into one factor score that was subsequently used in the

MANOVA to detect grade differences. However, previous research with older students suggests that

development of clause density in younger students consists of multiple periods of slower, sometimes

plateaued growth prior to the eighth grade, and may be a useful measure for detecting grade effects in

later years (Nippold, Ward-Lonergan, & Fanning, 2005). Further studies may wish to compare

measures of grammatical complexity to determine their utility for detecting significant differences

between elementary grade level groups.

No grade level differences were indicated for either grammatical accuracy, or lexical diversity.

The lack of grade differences for grammatical accuracy is similar to Puranik et al., although they utilized

different measures for this dimension of microstructure (percentage of grammatically correct T-units,

proportion of spelling errors, conventions). This finding also is similar to Nelson and Van Meter’s (2007)

finding of no main effects for grammatical error rates among grades one through five. In the present

investigation, grammatical accuracy was measured by the percentage of grammatically correct

sentences and the total number of grammatical errors. Puranik et al. posited that older children may

attempt to produce more complex sentence structures, and as a result could generate more errors than

expected, making their grammatical accuracy scores more parallel with those of younger participants.

When examining the grade level means for grammatical accuracy for the present sample, even though

the difference did not reach statistical significance, fourth grade showed lower factor score values in

both genres (narrative M = -.23; expository M = -.10) than second (narrative M = .14; expository M =

.18) and third grades (narrative M = .08; expository M = -.05), possibly supporting Puranik et al’s

suggestion. Tables 10 and 11 provide further evidence of depressed grammatical accuracy levels in

fourth grade, particularly with respect to the total grammatical errors produced. Additionally, even

though there was no significant correlation detected across all three grade levels for either narrative (r =

-.04) or expository (r = .19) samples, correlations among grammatical complexity and grammatical

accuracy within each grade level (Table 16) suggest this relationship warrants further exploration.

Therefore, future studies may want to increase the number of items, or variables, measured for both

grammatical complexity and accuracy factors and compare the relations among them under varying

contexts and task demands.

54

Effect of Genre on Microstructure

It was anticipated that performance on microstructure measures would be better in the narrative

genre, particularly for students in the second, and possibly the third grades, who were assumed to

possess less knowledge of and experience with expository discourse structures. Results of a series of

repeated measures ANOVAs rejected this hypothesis, indicating that mean performance on all four

microstructure measures was rather stable within grades and across genres. This finding initially

appeared to deviate from previous studies that indicated greater productivity and grammatical

complexity in a narrative versus an expository genre among elementary school children (Berman &

Verhoeven; Scott & Windsor, 2000); although, a direct comparison to second graders in the present

study cannot be made as Berman and Verhoeven and Scott and Windsor’s samples did not include

second graders (their participants were fourth and fifth grades). Visual analysis of the grammatical

complexity results (Table 17) for second grade in the present sample suggested a higher mean value in

the expository genre than in the narrative genre (narrative M = -.15; expository M = .45), which is in

contrast to the hypothesis calling for greater values in the narrative genre (Scott & Windsor, 2000).

Furthermore, visual analysis of fourth grade results actually supported the hypothesis, suggesting a

higher mean value for narrative grammatical complexity (narrative M = .03; expository M = -.24).

However, neither of these differences was statistically significant. The finding of no genre effect for

either lexical diversity or grammatical accuracy is similar to previous findings for elementary students

(Berman & Verhoeven, 2002; Scott & Windsor 2000). As with previous samples examined, the

developmental progression of lexical diversity and grammatical accuracy may be slower overall,

regardless of genre, and therefore, more challenging to detect differences between subsequent

elementary grades.

Effect of Grade Level on Macrostructure

The second aim of this study was to determine whether there were differences among grades

and between genres in macrostructure elements. It was expected that differences would be detected

between grade levels based on measures of organization, coherence, and text structure, with the oldest

students (fourth graders) demonstrating the highest levels of performance in this regard. As

hypothesized, MANCOVA detected grade level differences in both genres for macrostructure between

second and third grades, and between second and fourth grades. Second graders scored lower on

narrative and expository organization, text structure, and cohesion than third and fourth graders (see

Tables 12 and 13). However, in contrast to expectations, third and fourth grade did not differ

significantly from each other in either genre. In examining the grade level means in the narrative genre

for the three variables constituting the macrostructure measure, it was noted that the third grade mean

was higher than fourth in text structure (third grade M = 2.76; fourth grade M = 2.52) and cohesion

55

(third grade M = 2.0; fourth grade M = 1.93), although these differences were not substantial enough to

reach statistical significance. In the expository genre, grade level means indicated a slight (but

insignificant) difference between third and fourth grades on organization (third grade M =4.66; fourth

grade M = 4.85), text structure (third grade M = 1.81; fourth grade M = 1.92), and cohesion (third grade

M = 1.34; fourth grade M = 1.46).

The grade level trends observed in the expository genre between second and third, and second

and fourth appear to be due to text structure scores more so than organization or cohesion scores. It

may be that an expository writing sample could be somewhat organized and cohesive, yet not reflect

the targeted genre structure. This is understandable, especially for second graders, whose knowledge

of genre-specific text structure is thought to be less established, especially for expository genre, than

for older students (Berman & Verhoeven, 2002; Nelson, et al, 2004). A single second grade participant

scored at a level 3 for expository text structure on the macrostructure rubric, whereas the remaining

second graders scored at level 1 (n=19) or level 2 (n=6). This was in contrast to the number of third

graders (n=5) and fourth graders (n=5) scoring at a level 3 or higher, and also was dissimilar from the

number of third and fourth graders scoring at a level 2 (3rd n = 15; 4th n = 13). Thus, the grade level

trends appeared to be impacted by floor effects for the second graders on measures of macrostructure.

In the future, researchers may want to develop more sensitive measures to examine children's

knowledge of genre-specific text structures across grade levels, particularly their knowledge of text

structure requirements for the genres most commonly assessed in the classroom, to help further define

this developmental progression.

In the narrative genre, third and fourth grade means for organization and cohesion were nearly

equivalent (mean difference for expository organization = .01; for expository cohesion = .04). A trend in

favor of higher scores for third graders on text structure was noted (mean difference from fourth = .24);

however, this difference was not substantial enough to reach statistical significance (see Figure 5). Cox

et al. (1990) detected significant differences between the third and fifth grades (favoring the fifth grade

participants) for cohesive harmony in a narrative, but not expository genre. Their findings, combined

with the current data, may suggest that the developmental progression of cohesion occurs along a

more gradual continuum, so that the changes due to increases in grade level are more easily detected

between non-subsequent grade levels.

To further explore the levels measured within the organization score, a MANOVA was

conducted post hoc with grade level as the between subjects factor and the 3 sub-scores

corresponding to the organization of the introduction, body, and conclusion as the dependent variables.

Again, a significant multivariate effect was noted for grade level. Although all 3 grade levels were

similar to each other in regard to their scores for narrative introductions, grade differences were noted

56

between second and third, and between second and fourth, for narrative body and conclusion scores. A

potential floor effect for second grade on body and conclusion scores was noted upon examination of

boxplots for these variables. Second graders seemed to incorporate stronger introductions than the

other portions of their text (i.e., body, conclusion), by incorporating more aspects of narrative

organizational structure in the beginning rather than in the remainder of their text. In light of previous

investigations that examined levels of story grammar reflected in children’s writing, none of the second

graders produced what would be considered a complete episode; at best, they incorporated one causal

sequence (i.e., a series of actions casually linked but without planning, and characterize most of the

story; Glenn & Stein, 1980, Nelson, Bahr, & Van Meter, 2004). Nelson et al. (2004) reported that

second graders would reasonably be expected to produce at least temporal sequences (i.e., a series of

actions that are temporally linked in a "what next" strategy, with ideas often linked by "and", "so", and

"then"). Thus, this suggests that the second graders in this study produced story grammar structure

consistent with developmental expectations. It is plausible that the macrostructure rubric, even though it

incorporated different criteria for narrative versus expository text, was not as sensitive to detect specific

instances of story grammar use as would a more in-depth scale of story grammar use (see Nelson et

al, 2004).

Third and fourth graders were similar in scores for narrative body and conclusion. In contrast to

second grade, causal sequences and a few abbreviated episodes (i.e., statement of the problem as

well as characters' aims or intentions are implied or stated in word choices such as "decided" and

"wanted to") occurred more frequently in the text produced by third and fourth graders. Complete

episodes were rarely evident. Laughton & Morris (1989) concluded that only about half of the third and

fourth graders in their sample produced complete stories based on story grammar markers. Within their

sample, Montague, Maddux, and Dereshiwsky (1990) did not find any significant differences between

fourth and fifth graders’ use of story propositions. Thus, the current findings mirror past results.

Additional analyses may be warranted in future investigations to code for specific story grammar

elements in the narrative writing produced by students across multiple grade levels to compare the

developmental progression of their use of narrative elements to development of macrostructural

elements of other emerging genre forms. Longitudinal investigations of this nature would advance the

literature base in this area.

No significant multivariate effect was detected for grade level in the expository genre. Reviewing

grade level means indicated a possible trend between second and fourth grades for the expository

body scores, with fourth graders receiving higher scores than second graders. It may be that the lack of

significant grade level differences reflects students' limited familiarity with, and use of, expository

organization and text structure, thereby offering further support to the assumption that young students,

57

including those in second through fourth grade, have less experience and knowledge of various text

structures and genre specific organization. These findings suggest they do not yet exhibit consistent

expository genre knowledge.

For each grade level, participants tended to score higher on the introduction than the body or

conclusion in both genres. Paired t-tests showed that for the entire sample, there were significant

differences between narrative and expository introduction (t = 6.66, p <.001), body (t = 3.58, p <.001),

and conclusion scores (t = 4.26, p <.001), favoring the narrative genre. A closer examination at each

grade level provided additional insight to differences between genres and grade levels. For example,

there was a significant difference for second grade between narrative and expository introduction

scores (t= 3.19 , p= .004; higher scores obtained in narrative genre). The finding of a significant genre

effect for second graders on the introductions of their texts provides additional support for the notion

that young children have more knowledge of narrative text structure and organization than expository

text structure. In contrast, no significant differences were noted for narrative and expository body or

conclusion scores for the second grade. Second graders appeared to produce less organized body and

conclusion sections in their text, regardless of genre. Significant differences were observed for third

grade for all 3 organization subscores, (introductions, t = 5.23, p <.001; body, t = 3.97, p <.001; and

conclusion t = 5.00, p<.001), favoring the narrative genre. For grade 4, significant differences were

observed between narrative and expository introductions (t = 2.98, p =.007), body (t = 2.49, p=.025),

and conclusion (t= 2.45, p=.022). These findings indicate that the third and fourth graders in the sample

produced more organized body and conclusion sections in their narrative text, again providing support

for the tenet favoring narrative knowledge over expository knowledge in younger children.

Effect of Genre on Macrostructure

In contrast to expectations, repeated measures ANOVA did not reveal any cross-genre

differences for performance on the macrostructure measure. In fact, for fourth grade, the mean

macrostructure values were very similar across genres. Visual analysis of the results for the narrative

genre indicated a sharp spike in macrostructure scores from second to third grades (see Figure 5),

although this difference was not statistically significant. The potential genre effects on macrostructure

were difficult to anticipate given the range of findings in previous investigations. However, it was

anticipated that performance on macrostructure measures would be similar in both genres for the oldest

students (fourth grade) based on the assumption that experience and knowledge of various text

structures and text cohesion increases with age.

The lack of significant differences in macrostructure scores between the narrative and

expository genres may be due to the type of writing prompts utilized to elicit the samples. It is possible

that the prompts selected may not have fully represented one genre versus another. However, a post

58

hoc analysis of writing samples via a genre identification task revealed that most of the samples

checked were coded reliably for genre type by naive raters. Thus, it appears the majority of writing

samples reflected a specific genre structure (narrative or expository). Nevertheless, some children may

have produced text in a structure other than the elicited genre. It may be these children utilized

inconsistent or limited genre structures in their writing that make it difficult to identify the genre structure

produced in the writing sample. The lack of significant differences in macrostructure also may be due to

the use of "hybrid" genres that incorporate a variety of text structure features from one or more genres.

In situations such as this, the scoring rubric designed for this investigation would need to be altered to

capture occurrences of “mixed” or “hybrid genres. Further exploration is warranted to establish how

adequately writing prompts elicit the intended genre.

Relations among Elements of Microstructure and Macrostructure

The third research question aimed to determine the relations among microstructure and

macrostructure elements in both genres. It was difficult to speculate the potential differences with a lack

of previous research specific to this area; although, researchers have suspected that a relationship did

indeed exist within the narrative genre (Mackie & Dockrell, 2004; Newcomer et al, 1987; Nodine et al,

1985). When examining oral and written narrative competence, researchers have demonstrated that

microstructural and macrostructural variables represent two distinct, underlying components, and that

narrative production involves a complex system of demands at the microstructural and macrostructural

levels (e.g., Justice, 2004). However, there is a paucity of studies delineating the relations among these

elements for genres other than narratives.

Correlational analyses revealed significant associations in both genres between productivity, a

microstructure measure, and performance on the macrostructure measure. Furthermore, significant,

small negative correlations were detected in the expository genre between macrostructure and

grammatical complexity and grammatical accuracy (i.e., as macrostructure value increased,

grammatical complexity and accuracy decreased). Results varied by grade. For example, within second

grade, a significant positive correlation was detected between narrative grammatical complexity and

macrostructure, and a significant negative correlation between expository grammatical accuracy and

lexical diversity (i.e., proportion of content words to total words). As such, this negative correlation

between these two factors of microstructure indicates that for second graders, as their proportion of

content words increased in the expository genre, the level of grammatical accuracy decreased. When

producing text, an individual writer must juggle the demands of grammatical and vocabulary constraints

(Justice, 2004). As discussed with the grammatical complexity factor, it may be that grammatical

accuracy represented a trade-off between the lexical and cognitive demands of the genre. However,

further exploration of this relationship in narratives and other genres is warranted to confirm the nature

59

of this relationship. Within third grade, a significant correlation was detected between productivity and

macrostructure in narrative and expository samples. Thus, regardless of genre, as third graders'

productivity levels increased, so did their macrostructure performance. Within fourth grade, a

significant correlation was found between expository productivity and grammatical complexity.

Therefore, for the fourth graders, students who were more productive in the expository genre tended

also to produce more complex grammatical structures.

In summary, for the expository genre, there was a significant, moderate negative correlation

between grammatical complexity and grammatical accuracy for second graders, and a significant

moderate positive correlation between productivity and grammatical complexity for fourth graders. No

significant correlations were noted between microstructure features for third graders. The patterns

observed within each grade level for correlations among different features of microstructure in

expository samples provide further evidence of linguistic trade-offs occurring for second and fourth

graders, and provide preliminary evidence of a genre effect on the trade-off phenomenon.

Effects of Ethnicity, Gender, and SES

Although not the primary focus of this investigation, the potential influences of other factors on

writing performance warranted some attention. There was no effect of ethnicity, gender, or SES for

dependent variables in either the narrative or expository genre. The finding of no effect of gender is in

contrast to the results of Nelson and Van Meter (2007), who detected a significant effect of gender in

favor of female participants on measures of productivity. However, it may be that the greater

homogeneity of the Nelson and Van Meter participants in varying ability groups influenced their result

for gender analyses because they did not hold constant any other factors such as typical versus

atypical development, nor did they control for reading levels. The results of the present study

corroborated Berman and Verhoeven’s findings of no significant gender effects for total words, lexical

diversity, or clause density in either narrative or expository genres. Even so, subsequent investigations

should continue to examine the role of gender in writing outcomes (Peterson, 2006). For example, for

older students, gender has been known to impact self-efficacy of writing, with girls demonstrating higher

levels of self-efficacy (Andrade, Wang, Du, & Akawi, 2009), and formation of writing goals, with boys

and girls approaching the writing task differently (task-related goals for girls, performance-related goals

for boys; Pajares, Britner, & Valiante, 2000).

Even though significant differences were not detected for SES, there are multiple reasons these

findings should be interpreted with caution. Regarding SES, a participant’s classification of lower SES

based on receiving F/RL is dependent on his or her parent or guardian submitting a voluntary

application through the school system. As such, there was no way to ascertain whether the higher SES

group (i.e., those not receiving F/RL) may have included students who would have qualified for F/RL,

60

but whose parents or guardians had simply not applied. Furthermore, this measure of SES only

consisted of two levels. Researchers may elect to incorporate more sensitive measures of SES (e.g.,

mother's education) in future investigations with additional levels of analysis (e.g., low, middle, high

SES; low-middle, middle, and upper-middle SES), and with consideration of specific home

environmental factors that may confound SES as a predictor of achievement (Purcell-Gates, 2000).

In addition to increasing the sensitivity of SES measures, consideration of other parental

characteristics that may affect language and literacy development of children is warranted (e.g.,

culture/ethnicity, parental beliefs). Wasik and Hendrickson (2004) proposed that in addition to parental

characteristics, a comprehensive model for the variables that affect children’s language and literacy

development include child characteristics, the home environment, and parent-child interactions. The

present investigation considered some child characteristics, and one parent characteristic (SES), but

did not consider these additional factors in the model. Researchers may desire to incorporate analyses

of these factors in their research designs in the future. Further investigation into the effects of SES on

dependent writing measures is warranted, particularly with larger samples to increase power to detect

medium and small effects, and allow additional possibilities with regard to inferential analysis and

interpretation. Although no single measure of SES has yet been highlighted as the premier choice,

measures of SES that go beyond family income should be employed in future studies that examine

achievement gaps (Craig, Zhang, Hensel, & Quinn, 2009).

The influence of cultural-linguistic factors on writing performance, such as ethnicity and dialect,

is important to consider. First, one must acknowledge that ethnicity and dialect are not equivalent, and

each factor warrants separate consideration. Further, in the present study, no measure of dialect was

administered. No effect of ethnicity was detected in the present sample. However, regardless of

reported ethnicity, it is possible that dialectal influences may exist for individual children, and could

affect the outcome measures of grammatical accuracy. The dialect shifting-reading achievement

hypothesis suggests that students who successfully shift from dialectal forms reflective of non-

mainstream English dialects (e.g., AAE) to Standard American English (SAE) forms in different literacy

tasks (including writing) demonstrate better reading outcomes than students who do not make the shift

as adequately (Craig et al., 2009). Investigations employing larger samples of participants with

ethnically diverse backgrounds, and incorporating distinct a priori measures of dialect (e.g., dialect

density measures), may have better chances of detecting possible differences. If differences are indeed

detected in this manner, investigators can recode the SALT files to capture features of a specific dialect

that has been observed in the sample (e.g., AAE). It would be worthwhile to compare results for written

grammatical and lexical microstructure variables, as well as text structure influences, of dialectal

speakers to capture the weight of influence that dialectal differences may exert on dependent writing

61

measures for both microstructure and macrostructure (Terry, 2006; Thompson, Craig, & Washington,

2004).

Limitations and Future Research

A variety of potential limitations to the present investigation have been presented throughout

this discussion. However, some additional considerations for future research are necessary to note.

One consideration is the method used to elicit writing samples employed in the current study. A single

elicitation technique was incorporated (i.e., response to writing prompt). It is important to consider that

grade and genre effects may vary as a result of differences in prompting procedures and targeted

genre structure (Scott, 1994). Moreover, time constraints may limit productivity, and possibly any

variables strongly correlated with productivity. The timeframe allowed for students to produce their

writing samples was consistent with regular classroom practices guided by statewide assessments.

Whether dissimilar results would be obtained with writing samples produced via different sampling

techniques is unknown. More work is needed in this area to compare the value of various elicitation

techniques to capture the possible relations between elicitation method and writing outcomes.

Furthermore, the degree of the relations among elements of microstructure and macrostructure may be

shaped by the actual genre structure produced, especially considering the results of the post hoc genre

identification task. Thus, investigators planning future studies in this are may elect to first establish the

reliability of selected prompts to elicit the intended genre, and plan in advance a post-hoc analysis to

verify the reliability of selected prompts within their sample.

Sensitivity of dependent measures to detect grade and genre differences is another crucial

element of this investigation to note in light of the findings. For example, potential floor and ceiling

effects were indicated for individual grades on a few dependent measures. As discussed at the

beginning of the results section, several variables, for second grade in particular, exhibited skewness

and/or kurtosis reflective of floor and ceiling effects. This may have limited the measure’s sensitivity to

detect differences between grade levels or to negatively impact the power of parametric analyses

utilized. However, MANOVA is reportedly robust against mild to moderate violations of normality

(Fields, 2005). Furthermore, these data points were considered to be accurate depictions of the range

of performance and true reflections of the variability inherent in young children’s performance on writing

measures. Larger samples may help allay this concern.

Regarding the sensitivity of the macrostructure measure, the fact that this measure resulted in a

unidimensional measure of macrostructure (according to the EFA results) would seem to be

contradictory to some authors’ recommendations against the use of holistic score ratings of writing

performance to inform instruction and monitor growth (Nelson & Van Meter, 2004). However, as noted

in the present investigation, a “holistic” rating scale for macrostructure was a useful method to compare

62

a particular student’s or grade level’s performance in comparison to peers or comparison groups. In

contrast, EFA results indicated that the microstructure measure consisted of four distinct factors. As

such, microstructure, in contrast to macrostructure, would be best examined with an analytic scoring

method, utilizing more than one factor or score. Either way, the purpose for the writing assessment, as

well as the reliability of a particular scale to fulfill that purpose, should be the focus at the outset. In

some states, including Florida, the statewide progress monitoring tool for writing in the elementary

grades (i.e., Writes Upon Request) is administered multiple times per school year and yields only a

holistic score. The raters consider four factors in their ratings of student text: focus, organization,

support, and conventions (FLDOE, 2009). Educators are first cautioned against using this single score

as the sole determinant of a student's writing proficiency, and encouraged to interpret this score in light

of the student's performance in other writing tasks and contexts (FLDOE).

Additional considerations for research design are warranted. The data examined in this

investigation represented the pre-test writing performance of students from three grade levels within

one school, who also were recruited to participate in a spelling intervention study. Even though the

design of the school sampled is intended to be representative of statewide student demographics,

analyses based on samples of convenience still may not be wholly representative of the general

population. Caution is therefore called for when applying these interpretations to samples other than

that included in this study. However, this investigation has demonstrated the utility of these measures

for detecting grade and genre differences, and can be utilized when developing local norms.

There are two notable concerns about the use of a covariate in the main analyses. Grade level

differences on the GRADE Comprehension Composite scores indicated the need to include the scores

as a covariate in the main analyses. However, even though there was a significant group difference

between third and fourth grades on this measure, the magnitude of this difference, as indicated by the

effect size, was small (.12). The clinical significance of this small difference is unclear, especially

considering that the mean scores for the fourth grade group remained well within the average range.

Furthermore, as some authors warn against employing ANCOVA with nonrandom groups, researchers

should consider possible alternatives to address the limitations involved and improve their designs

(Miller, 2001).

Data derived from educational research often have a nested structure (e.g., students are nested

in classes, classes are nested within schools, and schools are nested within school districts). With

consideration of the potential impact of quality of writing instruction on writing outcomes (Mehta,

Foorman, Branum-Martin, & Taylor, 2005; Moats, et al 2006), future researchers are encouraged to

utilize ANOVA designs with nested factors to detect within-class effects. The current design did not

allow for this level of analysis. An increased sample size, with a larger number of classrooms examined

63

at each grade level, and additional schools, would support nested designs to look at class/teacher

effects more specifically.

Results of the present investigation extend findings from previous studies, and add to the

existing literature regarding development of and relations among written microstructure and

macrostructure features within and across grade levels and genre types. The dependent measures

utilized in the present study have been suggested as not only useful tools for differentiating groups of

students on some factor of achievement (e.g., reading level, learning disability, language impairment;

see Nelson & Van Meter, 2007; Puranik et al., 2007), but also as useful progress monitoring tools for all

students. To date, there have been few longitudinal studies designed to examine the utility of these

measures to monitor student progress across multiple genres and grade levels.

Educational Implications

Educators are encouraged to consider the lack of differences between grade levels for some of

the dependent measures in light of established grade level expectations that are reflected in state

standards for writing. For example, Florida standards (FLDOE, 2009) require second graders to write in

a variety of informative and expository forms, and produce narratives based on real or imagined events

(including a main idea, characters, sequence of events, and descriptive details). Beginning in third

grade, students are expected to write short persuasive texts, write in additional varieties of informative

and expository forms, and produce more complex narratives (including additional items over and above

those listed for second grade: setting, plot, sensory details, and logical sequence of events).

Additionally, third graders are expected to produce a minimal expository structure containing at least

three paragraphs, and including a topic sentence, supporting details, and relevant information. By

fourth grade, the expectations are higher for persuasive (including use of persuasive techniques,

supporting arguments and detailed evidence), informative and expository (essays including

introductory, body, and concluding paragraphs), and narrative writing (all of the previously mentioned

components plus a context to enable the reader to imagine the world of the event or experience).

Based on review of the present data, it is clear that not all of the writing samples collected reflected

mastery of the previous grade level’s standards for writing. This begs the question, if the established

grade level expectations are considered reasonable, how well are current instructional practices

designed to support student achievement of these standards? Future research needs to determine the

extent to which writing instruction, assessment, and progress monitoring adhere to grade level

standards for writing performance. In the meantime, state writing standards that are being developed or

revised need to be research-based, and educators, researchers, and policy-makers need to work

collaboratively to design instruction that is reflective of research-based standards for writing.

64

In general, the importance of timely detection of students at risk for writing problems, and the

provision of early intervention for defined problems, is garnering increased attention in the educational

system in recent years (Singer & Bashir, 2004). Federal and state educational policy reflects this

movement to improve the writing proficiency of all students (Troia, 2009). However, writing research is

an extremely complex undertaking, and as such, requires careful planning and attention to

methodological implications and limitations of previous research. Much more is to be learned regarding

the development of linguistic features in children’s writing, the effects of multiple factors at the child,

family, classroom, and school levels, and development of reliable and valid writing assessment and

progress monitoring tools. Once writing problems are detected, and instruction or intervention is

planned and provided, reliable progress monitoring tools are necessary to document the student’s

response to the interventions implemented. When considering development of appropriate progress

monitoring tools for writing, one should consider that some measures are more sensitive for capturing

developmental progression within and between grade levels than others.

Conclusion

This study examined multiple dimensions of written language produced by children in grades 2,

3, and 4 in narrative and expository writing samples. The samples were analyzed for developmental

progression of linguistic elements of microstructure and macrostructure represented by the five factors

of productivity, grammatical complexity, grammatical accuracy, lexical diversity, and macrostructure.

Results of this study suggest that variables of written microstructure and macrostructure were sensitive

to grade and genre level differences, that productivity and macrostructure were related in both genres

for all three grade levels, and that one cannot assume the older students will outperform younger

students on all measures. This latter finding was thought to be due to a trade-off between linguistic and

cognitive demands. Consequently, future research needs to establish these trade-off trends in larger

samples and examine the effects of different academic contexts (e.g., variable elicitation techniques,

discourse structures, content specific assignments) on this phenomenon.

Acknowledging that writing is truly an essential component of literacy (broadly defined), and

therefore plays a hefty role in the national literacy crisis, then “poor facility in expressing thoughts

through written language” may persist as the “most prevalent disability of communication skills” (Lerner,

1976; p. 266). Given the importance of writing, researchers and practitioners have a responsibility to

persevere and continue meeting the demands of this challenge directly, through continual exploration of

various aspects of writing development and writing proficiency, with the ultimate goal to improve the

writing outcomes of all students.

65

APPENDIX A

CONSENT FORM & IRB APPROVAL

66

Office of the Vice President For Research Human Subjects Committee Tallahassee, Florida 32306-2742 (850) 644-8673 · FAX (850) 644-4392 APPROVAL MEMORANDUM Date: 9/18/2008 To: Kenn Apel [kenn.apel@comm.fsu.edu] Address: 1200 Dept.: COMMUNICATION DISORDERS From: Thomas L. Jacobson, Chair Re: Use of Human Subjects in Research The Effect of a Multiple-Linguistic Factor Spelling Approach on Spelling, Reading, and Writing Abilities The application that you submitted to this office in regard to the use of human subjects in the research proposal referenced above has been reviewed by the Human Subjects Committee at its meeting on 09/10/2008. Your project was approved by the Committee. The Human Subjects Committee has not evaluated your proposal for scientific merit, except to weigh the risk to the human participants and the aspects of the proposal related to potential risk and benefit. This approval does not replace any departmental or other approvals, which may be required. If you submitted a proposed consent form with your application, the approved stamped consent form is attached to this approval notice. Only the stamped version of the consent form may be used in recruiting research subjects. If the project has not been completed by 9/9/2009 you must request a renewal of approval for continuation of the project. As a courtesy, a renewal notice will be sent to you prior to your expiration date; however, it is your responsibility as the Principal Investigator to timely request renewal of your approval from the Committee. You are advised that any change in protocol for this project must be reviewed and approved by the Committee prior to implementation of the proposed change in the protocol. A protocol change/amendment form is required to be submitted for approval by the Committee. In addition, federal regulations require that the Principal Investigator promptly report, in writing any unanticipated problems or adverse events involving risks to research subjects or others. By copy of this memorandum, the Chair of your department and/or your major professor is reminded that he/she is responsible for being informed concerning research projects involving human subjects in the department, and should review protocols as often as needed to insure that the project is being conducted in compliance with our institution and with DHHS regulations. This institution has an Assurance on file with the Office for Human Research Protection. The Assurance Number is IRB00000446. Cc: Juliann Woods, Chair [jwoods@fsu.edu]

HSC No. 2008.1364

67

APPENDIX B

WRITING INSTRUCTIONS AND PROMPTS

“Today you are going to do two pieces of writing on topics I give you. I want you to do everything you

know how to do as a writer to complete this assignment. You may use any strategies you know that

help you. Let me read the first prompt to you. (Read prompt aloud.) When you are finished, you may

re-read your paper and make any changes you want.”

Narrative: "Tell me about a time that someone surprised you and what happened."

Expository: "Pretend you are a super hero and you are being interviewed on the news. Tell

everyone what special powers you would have. Also, explain what you would do with them to

help the world."

68

APPENDIX C

SALT PROTOCOL FOR MICROSTRUCTURE VARIABLES

Entering utterances (sentences from a writing sample) as T-units in SALT

• A t-unit is one main clause plus any subordinate (dependent) clause or nonclausal structure (such as a prepositional or verbal phrase) that is embedded in the main clause.

• A T-unit is an independent clause (a subject and a predicate) along with ay phrases or clauses embedded in it. • All coordinated clauses are separated out into T-units, unless they contain a co-referential subject deletion in the

second clause. Examples: 1. If people live in the city they don’t have to drive. (1 T-unit)

Enter is SALT as one line: If people live in the city they don’t have to drive.

2. There are people that live in the city and people that live in the country. (2 T-units)

Enter in SALT as two lines: There are people that live in the city. and people that live in the country. Examples of main clauses with embedded clauses: 3. Reading books is my favorite thing to do. (1 T-unit)

Enter is SALT as one line: Reading books is my favorite thing to do.

4. I like to read books. (1 T-unit) Enter is SALT as one line: I like to read books.

5. The book, which I forgot to bring, was my favorite. (2 T-unit) Enter is SALT as one line: The book, which I forgot to bring, was my favorite.

Example of a main clause with a phrase or clause subordinated to it: 6. She thanked me when I gave her the book. (1 T-unit)

Enter is SALT as one line: She thanked me when I gave her the book.

Example of 2 utterances that convey the same information but in different numbers of T-units:

69

7.I forgot the book but I can bring it tomorrow and I will give it back so I hope that is okay. (4 T-units)

Enter is SALT as four lines: I forgot the book. but I can bring it tomorrow. and I will give it back. so I hope that is okay.

8. I forgot the book that I need to give back, although I am coming tomorrow and can bring it then. (1 T-unit)

Enter is SALT as one line/utterance: I forgot the book that I need to give back, although I am coming tomorrow and can bring it then.

Example of a relative clause: 9. It was the boy who probably did it. (1 T-unit)

Enter is SALT as one line/utterance: It was the boy who probably did it. Example of an expanded noun phrase: 10. The large, green hairy monster ate the food. (1 T-unit)

Enter is SALT as one line/utterance: The large, green hairy monster ate the food.

Example of a nonfinite clause: 11. Keeping the room clean was her responsibility. (1 T-unit)

Enter is SALT as one line/utterance: Keeping the room clean was her responsibility.

Example of adverbial fronting: 12. There sat the big king elephant. (1 T-unit)

Enter is SALT as one line/utterance: There sat the big king elephant.

Other examples: She hid and forgot about it. (1 T-unit) - compound verb phrase After she hid it, she forgot where. (1 T-unit) -subordinated clause She decided to look for it. (1 T-unit) -verb as secondary verb, or nonfinite verb as an infinitive Looking for it proved difficult. (1 T-unit) -verb used as a noun phrase: gerund Art is fun because we paint but when we come back to our classroom we do our work. (2 T-units)

70

Enter in SALT as two lines: Art is fun because we paint. but when we come back to our classroom we do our work. I am 8 years old and I am in the third grade. (2 T-units)

Enter in SALT as two lines: I am 8 years old. and I am in the third grade. The boy who is my friend started working after I was done. (1 T-unit)

Enter in SALT as one line: The boy who is my friend started working after I was done.

He saw her and then he decided to turn around. (2 T-units)

Enter in SALT as two lines: He saw her. and then he decided to turn around.

She putted the dog’s collar on but he shook it off. (2 T-units)

Enter in SALT as two lines: She putted the dog’s collar on. but he shook it off.

Notes: • Young children tend to string together independent clauses with the coordinating conjunctions and, but, or, so. Use

of these conjunctions often signifies a new T-unit. • Each independent clause in a run-on sentence is counted as a separate T-unit.

SALT RULES:

• Must have a period at the end of each T-unit. • Always save the file you have segmented into T-units as a new file, different from the “original” SALT file, with the

word “Tunit” in the file name, save in the folder with your name.

Coding for Clauses in SALT Focus A: Coding for number of clauses per T-unit (clause density) Definition of Clause = group of related words with a subject and verb (some clauses are dependent, can stand alone, others are independent, cannot stand alone). 1.Using your checklist of participant files to code, open the file under your “Tunits” folder on the projects drive. 2. Enter the following codes at the top of every transcript: +[1CL]: 1 clause +[2CL]: 2 clauses +[3CL]: 3 clauses +[4CL]: 4 clauses +[MCL]: > 4 clauses (multiple) (*tip = you may want to copy and paste these codes from the first transcript to the others) 3. Count the number of clauses per T-unit and indicate at the end of the line (before the period) how many clauses you counted.

71

4. After you’ve entered the clause code for each T-unit in the transcript, please save the file under your file for “Clauses” on the projects drive. ------------------------------------------------------------------------------------------------------------ Focus B: Counting the number of clauses per sentence (for variables related to sentence complexity). 1. Using your checklist of participant files to code, open the file under your “Clauses” folder on the projects drive. 2. Count the number of clauses per sentence, and indicate at the end of the sentence (before the last period) how many clauses you counted. Remove the Clause counts that were previously counted at the T-unit level. Your final transcript document should only show the clause count per sentence. Note:

• There can be more than 1 T-unit in a sentence. • A sentence is generally based on the child’s punctuation in the original sample. (Sometimes kids forget to put a

period or some other punctuation at the end of a sentence. However, if they begin the next sentence with a capital letter, then it is treated in SALT as 2 sentences despite lacking punctuation in the original paper).

3. After you’ve entered the clause code for each sentence, please save the file under your file for “Sentence clauses” on the projects drive.

Coding for Grammatical Errors (GE) Add the following code to the list of codes at the top of every transcript/SALT file: +[GE]: Grammatical Error Grammatical errors = verb or pronoun tense/agreement/case, omitted or incorrect inflection / omitted or substitution of grammatical elements, violation of word order. Examples:

• The city have a lot of people. (problem = subject/verb agreement) How it should be coded in SALT:

The city have(GE) a lot of people.

• People living in the country are also close to their jobs, which are usually farming. (problem = unclear referent)

How it should be coded in SALT: People living in the country are also close to their jobs, which(GE) are usually farming.

• There are a lot of farmers in suburbs. (problem = missing article before “suburbs”) How it should be coded in SALT: There are a lot of farmers in(GE) suburbs. • There is a lot of schools. (problem = subject/verb agreement) How it should be coded in SALT: There is(GE) a lot of schools.

Coding for Simple/Complex and Correct/Incorrect Sentences Add the following codes to the list of codes at the top of every transcript/SALT file: +[SC]: Simple Correct +[CC]: Complex Correct

72

+[SI]: Simple Incorrect +[CI]: Complex Incorrect

• Correct Sentence = no grammatical errors (no GE codes) • Incorrect Sentence = has one or more grammatical errors (1 or more GE codes)

• Simple Sentence = A sentence with only one main clause.

Examples of Simple Correct Sentences [SC] People live in different places [1CL][SC] Farmers raise cows, pigs, and chickens [1CL][SC] Examples of Simple Incorrect Sentences [SI] There is[GE] a lot of schools, offices, and factories in the cities [1CL][SI]. There are a lot of farmers in[GE] suburbs [1CL][SI].

• Complex Sentence = A sentence with either: • one main clause and one or more subordinate/embedded clauses, or • two main clauses, or • one main clause and verb phrase joined by a coordinating conjunction (clause = must have a verb).

*Recall that a clause is a group of related words containing a subject with a verb. Examples of Complex Correct Sentences [CC] The suburbs are more crowded than the country but less crowded than the city [2CL][CC]. If people don’t want to drive a long way to their jobs, they live in the city [2CL][CC]. Examples of Complex Incorrect Sentences [CI] The farmers grows[GE] crops and give them to their animals [2CL][CI]. If people don’t want to drive a long way to their jobs, they lives[GE] in the city [2CL][CC].

SAVE YOUR CODING FILES WITHIN THE “SENTENCE TYPE” FOLDER

73

APPENDIX D

PROTOCOL FOR MACROSTRUCTURE VARIABLES

74

75

76

77

REFERENCES Andrade, H. L., Wang, X., Du, Y., & Akawi, R. (2009). Rubric-referenced self-assessment and self-

efficacy for writing. Journal of Educational Research, 102(4), 287-301. Ball, A. (2006). Teaching writing in culturally diverse classrooms. In C. A. MacArthur, S. Graham, & J. Fitzgerald (Eds.), Handbook of writing research (pp. 293-310). New York: Guilford Press. Barenbaum, E., Newcomer, P., & Nodine, B. (1987). Children's ability to write stories as a

function of variation in task, age, and developmental level. Learning Disability Quarterly, 10(3), 175-188.

Berman, R. A., & Verhoeven, L. (2002). Cross-linguistic perspectives on the development of text-production abilities: Speech and writing. Written Language and Literacy, 5(1), 1-43.

Biancarosa, C., & Snow, C. E. (2006). Reading next: A vision for action and research in middle

and high school literacy:A report to Carnegie Corporation of New York (2nd ed.). Washington, DC:Alliance for Excellent Education.

Blair, T. K., & Crump, W. D. (1984). Effects of discourse mode on the syntactic complexity of

learning disabled students' written expression. Learning Disability Quarterly, 7(1), 19-29. Borman, G. D., Dowling, N. M., & Schneck, C. (2008). A multisite cluster randomized field trial of Open Court Reading. Educational Evaluation and Policy Analysis, 30(4), 389-407. Cox, B. E., Shanahan, T., & Sulzby, E. (1990). Good and poor elementary readers' use of

cohesion in writing. Reading Research Quarterly, 25(1), 47-65. Craig, H. K., Zhang, L., Hensel, S. L., & Quinn, E. J. (2009). African American English-

Speaking students: An examination of the relationship between dialect shifting and reading outcomes. Journal of Speech, Language, and Hearing Research, 52, 839-855.

Crawford, L., Helwig, R., & Tindal, G. (2004). Writing performance assessments: How

important is extended time? Journal of Learning Disabilities, 37(2), 132-142. Davis, N., & Compton, D. L. (2008, March). Falling through the cracks: Children who are exceptions to the RtI identification process. Perspectives on Language Learning and Education, 15, 41-45. Dockrell, J. E., Lindsay, G., Connelly, V., & Mackie, C. (2007). Constraints in the production of

written text in children with specific language impairments. Exceptional Children, 73(2), 147-164.

Donovan, C. A., & Smolkin, L. B. (2006). Children's understanding of genre and writing

development. In C.A. MacArthur, S. Graham, & J. Fitzgerald (Eds.), Handbook of writing research (pp. 131-143). New York: Guilford Press.

Dunn, L. M., & Dunn, D. M. (2007). Peabody Picture Vocabulary Test - 4th ed. Minneapolis, MN:

NCS Pearson Assessments. Englert, C. S., Raphael, T. E., Anderson, L. M., Gregg, S. L., & Anthony, H. M. (1989).

78

Exposition: Reading, writing, and the metacognitive knowledge of learning disabled students. Learning Disabilities Research, 5(1), 5-24.

Englert, C. S., & Thomas, C. C. (1987). Sensitivity to text structure in reading and writing: A

comparison between learning disabled and non-learning disabled students. Learning Disability Quarterly, 10(2), 93-105.

Fields, A. (2005). Discovering statistics using SPSS. Second edition. London: Sage Publications. Fitzgerald, J., & Shanahan, T. (2000). Reading and writing relations and their development. Educational Psychologist, 35(1), 39-50.

Fey, M., Catts, H., Proctor-Williams, K., Tomblin, B., & Zhang, X. (2004). Oral and written

story composition skills of children with language impairment. Journal of Speech, Language, and Hearing Research, 47, 1301-1318.

Florida Department of Education (FLDOE, 2009). Florida Writes. Retrieved November 1, 2008,

from http://www.fldoe.org/asp. Glenn, C. G., & Stein, N. L. (1980). Syntactic structures and real world themes in stories

generated by children. (Technical report). Urbana, IL: University of Illinois, Center for the Study of Reading.

Graham, S., & Perin, D. (2007). Writing next: Effective strategies to improve writing of

adolescents in middle and high schools – A report to the Carnegie Corporation of New York. Washington, DC: Alliance for Excellent Education.

Graham, S., & Harris, K.R. (2003). Students with learning disabilities and the process of writing: A meta-analysis of SRSD Studies. In H.L. Swanson, K. Harris, and S. Graham (Eds.), Handbook of learning disabilities (pp. 323-344). New York: Guilford.

Houck, C., & Billingsley, B. (1989). Written expression of students with and without learning disabilities: Differences across the grades. Journal of Learning Disabilities, 22, 561-565.

Hunt, K.W. (1966). Recent measures in syntactic development. Elementary English, 43,732-739. Jenkins, J. R., Johnson, E., & Hileman, J. (2004). When is reading also writing: Sources of

individual differences on the new reading performance assessments. Scientific Studies of Reading, 8,125-151.

Kazdin, A.E. (2003). Research design in clinical psychology (4th ed.). Boston, MA:

Allyn and Bacon. Kennedy, P. (1993). Preparing for the twenty-first century. New York: Random House. Kidder, C. L. (1974). Using the computer to measure syntactic density and vocabulary density in

the writing of elementary school children. Unpublished doctoral dissertation, Pennsylvania State University, 1974.

Justice, L. (2004). The connection between oral narrative and reading problems: What’s the story? TEMPO Weekly Reader, 7.2, 2-9.

79

Laughton, J., & Morris, N. (1989). Story grammar knowledge of learning disabled students.

Learning Disabilities Research, 4, 87-95. Leedy, P. D., & Ormond, J. E. (2005). Practical research: Planning and design. Upper

Saddle River, NJ: Pearson Education, Inc. Lerner, J. W. (1976). Children with learning disabilities: Theories, diagnosis, teaching,

strategies. Boston: Houghton Mifflin. Li, Y. (2000). Linguistic characteristics of ESL writing in task-based e-mail activities. System, 28(2),

229-245. MacArthur, C. A., Graham, S., & Fitzgerald, J. (2006). Handbook of writing research. New

York: Guilford Press. MacWhinney, B. (1995). The CHILDES project. Hillsdale, NJ: Erlbaum. Mackie, C., & Dockrell, J. E. (2004). The nature of written language deficits in children with

SLI. Journal of Speech, Language, and Hearing Research, 47, 1469-1483. Mehta, P. D., Foorman, B. R., Branum-Martin, L., & Taylor, W.P. (2005). Literacy as a

unidimensional multilevel construct: Validation, sources of influence, and implications in a longitudinal study in grades 1 to 4. Scientific Studies of Reading, 9(2), 85-116.

Miller, G. A., & Chapman, J. P. (2001). Misunderstanding analysis of covariance. Journal of

Abnormal Psychology, 110(1), 40-48. Miller, J., & Chapman, R. (2005). Systematic Analysis of Language Transcripts (v.8). Madison,

WI: Waisman Center, University of Wisconsin-Madison. Moats, L., Foorman, B., & Taylor, P. (2006). How quality of writing instruction impacts high-

risk fourth graders' writing. Reading and Writing, 19, 363-391. Montague, M., Maddux, C., & Dereshiwsky, M. (1990). Story grammar and comprehension and

production of narrative prose by students with learning disabilities. Journal of Learning Disabilities, 23, 190-196.

Moore-Brown, B. J., Montgomery, J. K., Bielinski, J., & Shubin, J. (2005). Responsiveness to

intervention: Teaching before testing helps avoid labeling. Topics in Language Disorders, 25(2), 148-167.

Morris, N., & Crump, W. (1982). Syntactic and vocabulary development in the written language

of learning disabled and non-disabled students at four age levels. Learning Disability Quarterly, 5, 163-172.

National Center for Education Statistics (NCES) (2003). The nation’s report card: Writing 2002. Washington, DC: U.S. Department of Education.

Nelson, N. W., & Van Meter, A. M. (2002). Assessing curriculum-based reading and writing samples. Topics in Language Disorders, 22(2), 35-59.

80

Nelson, N. W., & Van Meter, A. M. (2007). Measuring written language ability in narrative

samples. Reading and Writing Quarterly, 23(3), 287-309. Nelson, N. W., Bahr, C. M., & Van Meter, A. M. (2004) The writing lab approach to language

instruction and intervention. Baltimore, MD: Paul H. Brookes. Newcomer, P. L., Barenbaum, E. M., & Nodine, B. F. (1988). Comparison of the story

production of LD, normal-achieving, and low-achieving children under two modes of production. Learning Disability Quarterly, 11(2), 82-96.

Newcomer, P. L., & Curtis, D. (1984). Diagnostic Achievement Battery. Austin, TX: Pro-Ed. Nippold, M., Ward-Lonergan, J., & Fanning, J. (2005). Persuasive writing in children, adolescents, and

adults: A study of syntactic, semantic, and pragmatic development. Language, Speech, and Hearing Services in Schools, 36, 125-138.

No Child Left Behind Act (NCLB) of 2001. (P.L.107_110 [20 U.S.C. 7801]). Nodine, B., Barenbaum, E., & Newcomer, P. (1985). Story composition by learning disabled,

reading disabled, and normal children. Learning Disability Quarterly, 8, 167-179. Pajares, F., Britner, S., & Valiante, G. (2000). Relations between achievement goals and self-beliefs of

middle school students in writing and science. Contemporary Educational Psychology, 25, 406- 422.

Persky, H. R., Daane, M. C., & Jin, Y. (2003). The nation’s report card: Writing 2002. (NCES

2003-529.) U.S. Department of Education. Institute of Education Sciences. National Center for Education Statistics. Washington, DC: Government Printing Office.

Peterson, S. (2006). Influence of gender on writing development. In C.A. MacArthur, S.

Graham, & J. Fitzgerald (Eds.), Handbook of writing research (pp.311-323). New York: Guilford Press.

Purcell-Gates, V. (2000). Family literacy. In M. L. Kamil, P. B. Mosenthal, P. D. Pearson, & R.

Barr (Eds.), Handbook of reading research, volume II (pp. 853-870). Mahwah, NJ: Lawrence Erlbaum Associates, Inc.

Puranik, C. (2006). Expository writing skills in elementary school children from third through

sixth grades and contributions of short-term and working memory. Unpublished doctoral dissertation. University of Florida, 2006.

Puranik, C., Lombardino, L. J., & Altmann, L. J. (2007). Writing through retellings: an

exploratory study of language-impaired and dyslexic populations. Reading & Writing, 20. 251-272.

Puranik, C., Lombardino, L. J., & Altmann, L. J. (2008). Assessing the microstructure of written

language using a retelling paradigm. American Journal of Speech Language Pathology, 17. 107-120.

Sanacore, J., & Palumbo, A. (2009). Understanding the fourth grade slump: Our point of view. The

81

Educational Forum, 73, 67-74. Sanders, T. J. M., & Schilperood, J. (2006). Text structure as a window on the cognition of

writing: How text analysis provides insights in writing products and writing processes. In C.A. MacArthur, S. Graham, & J. Fitzgerald (Eds.), Handbook of writing research (pp. 386-402).

Scott, C. M. (1994). A discourse continuum for school-age students: Impact of modality and genre. In G. Wallach & K. Butler (Eds.), Language learning disabilities in school-age children and adolescents (pp. 219-252). New York: Merrill. Scott, C. M. (2009). Language-based assessment of written expression. In G. A. Troia (Ed.),

Instruction and assessment for struggling writers: Evidence-based practices (pp. 358-385). Scott, C., & Stokes, S. (1995). Measures of syntax in school-age children and adolescents.

Language, Speech, and Hearing Services in Schools, 26, 309-317. Scott, C., & Windsor, J. (2000). General language performance measures in spoken and written

narrative and expository discourse of school-age children with language learning disabilities. Journal of Speech Language and Hearing Research, 43. 324-339.

Shanahan, T. (2006). Relations among oral language, reading, and writing development. In C.A.

MacArthur, S. Graham, & J. Fitzgerald (Eds.), Handbook of writing research (pp.171-186).

Singer, B. D. (2007). Assessment of reading comprehension and written expression in adolescents and adults. In A. G. Kamhi, J. J. Masterson, & K. Apel (Eds.), Clinical decision making in developmental language disorders (pp. 77-98).

Singer, B. D., & Bashir, A. S. (2002). EmPOWER: A strategy for teaching expository writing.

Boston, MA: Innovative Learning Partners, LLC. Singer, B. D. & Bashir, A. S. (2004). Developmental variations in writing composition skills. In

C.A. Stone, E.R. Silliman, B.J. Ehren, & K. Apel (Eds). Handbook of language and literacy development and disorders. (pp. 559-582). NY: Guilford Press.

Smith, M. C. (2000) What will be the demands of literacy in the workplace in the next

millennium? Reading Research Quarterly, 35(3), 378-383. SPSS Inc. (2009). Predictive Analytics Software for Windows version 17.0, Chicago, IL:

Authors. Stevens, J. (1997). Applied multivariate statistics for the social sciences. Third edition. Mahwah,

NJ: Lawrence Erlbaum Associates, Publishers. Terry, N. P. (2006). Relations between dialect variation, grammar, and early spelling skills.

Reading and Writing, 19, 907-931. Thompson, C. A., Craig, H. K., & Washington, J. A. (2004). Variable production of African

American English across oracy and literacy contexts. Language, Speech, and Hearing Services in Schools, 35, 269-282.

Troia, G. A. (2009). Instruction and assessment for struggling writers: Evidence-based

82

practices. New York: Guilford Press. U.S. Department of Education. (2008). Reading First. Retrieved November 1, 2008, from http://www.ed.gov/programs/readingfirst/index.html. Warner, R. M. (2008). Applied statistics: From bivariate through multivariate techniques. Los

Angeles: Sage Publications. Wasik, B. H., & Hendrikson, J. S. (2004). Family literacy practices. In C.A. Stone, E.R.

Silliman, B.J. Ehren, & K. Apel (Eds). Handbook of language and literacy development and disorders. (pp. 154-174). NY: Guilford Press.

Williams, K. T. (2001). GRADE. Circle Pines, MN: AGS Publishing, Inc. Zevengergen, A. A., Whitehurst, G. J., & Zevenbergen, J. A. (2003). Effects of a shared-reading intervention on the inclusion of evaluative devices in narratives of children from low-income families. Applied Developmental Psychology, 24, 1-15.

83

BIOGRAPHICAL SKETCH

Shannon Hall-Mills is a Doctoral Candidate in the School of Communication Science and Disorders at

Florida State University. Shannon was born and raised in Volusia County, Florida. She received her

Bachelor’s and Master’s degrees in Communication Disorders from Florida State University in 1999 and

2001. She obtained the Certificate of Clinical Competence in Speech-Language Pathology (CCC-SLP)

from the American Speech Language and Hearing Association. She maintains licensure to practice

Speech-Language Pathology with the Florida Department of Health, and certification from the Florida

Department of Education to work with students (K-12) who are speech/language impaired. Before

entering the doctoral program, Shannon worked with students in grades PK-5 as a school-based

speech-language pathologist and language diagnostician. While working on her doctorate, Shannon

was sponsored through a language and literacy leadership doctoral assistantship and graduate

scholarships from the Kappa Kappa Gamma Foundation, including a scholarship sponsored through

the Gates Foundation. Additionally, she served as an independent contracting therapist and Medicaid

provider for an early intervention program in the Tallahassee area. Shannon’s professional interests are

language and literacy development and disorders, school-based speech-language pathology services,

educational policy, and evidence-based practices. She is active in the American Speech Language

Hearing Association and the Florida Association of Speech-Language Pathologists and Audiologists.

Shannon currently resides in Tallahassee with her husband, serves as the state education consultant

for school-based SLPs in Florida through the Florida Department of Education, and continues to teach

at FSU in the School of Communication Science and Disorders and collaborate on language and

literacy research projects.

84