should students be allowed to use a word processor for the state writing assessment? pte 893 –...

Should Students Be Allowed to Should Students Be Allowed to Use a Word Processor for the Use a Word Processor for the

State Writing Assessment?State Writing Assessment?

PTE 893 – Leadership in Instructional PTE 893 – Leadership in Instructional TechnologyTechnology

Final Research ProjectFinal Research Project

Nancy EdwardsNancy Edwards

Table of ContentsTable of Contents TrendsTrends Reasons Computers Help Improve WritingReasons Computers Help Improve Writing Issues in Teaching Writing Issues in Teaching Writing Effects of Writing with ComputersEffects of Writing with Computers Importance of High Stakes TestingImportance of High Stakes Testing What research Says What research Says Issues Affecting Scores Issues Affecting Scores Reasons to Use Computers in TestingReasons to Use Computers in Testing TransitioningTransitioning Ways to Bridge the GapWays to Bridge the Gap ReferencesReferences

Trends in EducationTrends in Education

Trend #1Trend #1: The presence of : The presence of computers in schools has computers in schools has increased exponentiallyincreased exponentially Teachers use computers to Teachers use computers to

teach writingteach writing ““Regular use of computers for Regular use of computers for

writing over extended periods of writing over extended periods of time can lead to significant time can lead to significant improvements in student writing improvements in student writing skills” (Russell & Plati, 2001)skills” (Russell & Plati, 2001)


Trend #2Trend #2: : State testing State testing programs for increased programs for increased accountability have been put accountability have been put in place, in some form, in all in place, in some form, in all 50 states.50 states. Most writing assessments ban Most writing assessments ban

the use of computers and word the use of computers and word processors for the completion processors for the completion of these tests, except for of these tests, except for special needs students (NE special needs students (NE state update 10)state update 10)


ConclusionConclusion: These two trends : These two trends seem to be working against seem to be working against each othereach other


As the use of computers in schools, As the use of computers in schools, homes, and businesses increases, homes, and businesses increases, at least two problems arise:at least two problems arise:

1.1. For students who do most of their For students who do most of their writing via computer, writing longhand writing via computer, writing longhand is an artificial rather than a real world is an artificial rather than a real world task.task.

2.2. Paper and pencil tests may yield Paper and pencil tests may yield underestimates of the actual abilities underestimates of the actual abilities of students who are accustomed to of students who are accustomed to writing via computerwriting via computer

Trends in EducationTrends in Education ““The situation is analogous to The situation is analogous to

testing the accounting skills of testing the accounting skills of modern accountants, but modern accountants, but restricting them to the use of an restricting them to the use of an abacus for calculationsabacus for calculations.” (Russell & .” (Russell & Haney, 2000)Haney, 2000)

Reasons Why Computers May help Reasons Why Computers May help Improve Student WritingsImprove Student Writings

Computers help withComputers help with1.1. Generating textGenerating text

Students can easily create neat, Students can easily create neat, formatted textformatted text

Recopying is not necessary when Recopying is not necessary when making revisionsmaking revisions

Students can focus on the flow of Students can focus on the flow of thoughts rather than on grammar, thoughts rather than on grammar, spelling, and punctuation.spelling, and punctuation.


Computers help withComputers help with1.1. Generating textGenerating text2.2. Revising TextRevising Text

Neat appearance makes it easier to Neat appearance makes it easier to critiquecritique

Ability to alter text while preserving Ability to alter text while preserving appearanceappearance

Impermanence helps students see Impermanence helps students see writing as a fluid processwriting as a fluid process

Students who write on computer Students who write on computer return more often to their text and return more often to their text and make more revisions (Russell and make more revisions (Russell and Plati, 2001)Plati, 2001)



2.2. Revising TextRevising Text

3.3. Collaboration and AudienceCollaboration and Audience Easy to share writing on screenEasy to share writing on screen Computer itself may act as an Computer itself may act as an

audienceaudience



2.2. Revising TextRevising Text

3.3. Collaboration and AudienceCollaboration and Audience

4.4. FeedbackFeedback Spelling and grammar checkersSpelling and grammar checkers

Issues Which Deserve Attention from Issues Which Deserve Attention from Those Who Design Writing Programs Those Who Design Writing Programs

Utilizing New TechnologiesUtilizing New Technologies According to According to The Educational The Educational

Technology Journal Technology Journal , (1991) issues which deserve attention from those who design writing programs utilizing new technologies include:

IssuesIssues

#1#1. Has the school identified in explicit . Has the school identified in explicit terms why and how the word terms why and how the word processor might radically alter the processor might radically alter the quality of student writing?quality of student writing? Word processor provides greater fluidity Word processor provides greater fluidity

and flexibility than other writing and flexibility than other writing technologiestechnologies

Word processor makes thinking more Word processor makes thinking more powerful, as long as students are taught powerful, as long as students are taught how to use it in that wayhow to use it in that way

IssuesIssues

#2#2. . Is the staff prepared to employ Is the staff prepared to employ the word processor as an idea the word processor as an idea processor?processor? In-services will probably be a In-services will probably be a

requirementrequirement

IssuesIssues

#3#3. . Do students have sufficient Do students have sufficient access to computers so they can access to computers so they can develop their ideas appropriately develop their ideas appropriately over time?over time?

IssuesIssues

#4#4. . Is form more important than Is form more important than substance?substance? Stop confusing quantity with Stop confusing quantity with

qualityquality Good writing is complete when the Good writing is complete when the

ideas have been expressed with ideas have been expressed with clarity and impactclarity and impact

IssuesIssues

#5#5. . How will we know that writing How will we know that writing is improving?is improving?

Effects of Using Computers to Effects of Using Computers to Write Compositions Write Compositions

(as opposed to paper-and pencil)(as opposed to paper-and pencil) According to a study by Russell According to a study by Russell

(1999)(1999) Compositions on computer Compositions on computer

produced longer responses that produced longer responses that received higher scoresreceived higher scores

Students made more revisionsStudents made more revisions Compositions were of better qualityCompositions were of better quality Students reported very positive Students reported very positive

attitudes toward computer based attitudes toward computer based editing and writingediting and writing

Importance of High Stakes TestingImportance of High Stakes Testing

High stakes testing is used toHigh stakes testing is used to Promote students to higher gradesPromote students to higher grades Grant high school diplomasGrant high school diplomas Identify and sanction or reward low and Identify and sanction or reward low and

high performing schoolshigh performing schools

What Research Says About Using What Research Says About Using Computers as Opposed to Pencil-Computers as Opposed to Pencil-

and-Paper Testing for Writingand-Paper Testing for Writing

One study (Russell&Plati, 2002) One study (Russell&Plati, 2002) found that using pencil and paper, found that using pencil and paper, for students accustomed to for students accustomed to writing using a computer, writing using a computer, underestimated their underestimated their performance level for four to eight performance level for four to eight points on an eighty point scale.points on an eighty point scale.



Another study (Russell& Haney, Another study (Russell& Haney, 2000) showed that when 2000) showed that when students wrote on paper, only students wrote on paper, only 30% performed at a “passing” 30% performed at a “passing” level; when they wrote on level; when they wrote on computer, 67% “passed.”computer, 67% “passed.”



Because nearly ten million students Because nearly ten million students took some type of state sponsored took some type of state sponsored writing test last year, and over half of writing test last year, and over half of the students nationwide use word the students nationwide use word processors in school, these results processors in school, these results suggest that state paper and pencil suggest that state paper and pencil tests may be underestimating the tests may be underestimating the abilities of millions of students abilities of millions of students annually.annually.

Issues Affecting ScoresIssues Affecting Scores

Achievement scores on Writing Achievement scores on Writing Tests completed on the Tests completed on the computer are affected bycomputer are affected by#1. #1. How much practice students How much practice students

have had writing on the computerhave had writing on the computer


Achievement scores on Writing Achievement scores on Writing Tests completed on the Tests completed on the computer are affected bycomputer are affected by#2. #2. Keyboarding skillsKeyboarding skills

For students who keyboard 20 wpm or For students who keyboard 20 wpm or more, performing writing tests on more, performing writing tests on paper substantially underestimates paper substantially underestimates their level of achievementtheir level of achievement

For slower keyboarders, performing For slower keyboarders, performing these tests on computer adversely these tests on computer adversely affects their performance (Russell, affects their performance (Russell, 1999)1999)


Students do best when testing most Students do best when testing most nearly matches normal practicenearly matches normal practice

Until all students have access to and Until all students have access to and use computers regularly, open-use computers regularly, open-ended tests administered via a ended tests administered via a single medium will likely single medium will likely underestimate performance of underestimate performance of students accustomed to working in students accustomed to working in the alternate medium (Russell, the alternate medium (Russell, 1999)1999)

Reasons to Use Computers in Reasons to Use Computers in TestingTesting

Authenticity of testing would increaseAuthenticity of testing would increase Students would be better able to Students would be better able to

demonstrate their best workdemonstrate their best work Assessment would mirror real-world Assessment would mirror real-world

experienceexperience More valid measures of student More valid measures of student

writing would be available to judge writing would be available to judge student and school achievementstudent and school achievement

TransitioningTransitioning

As state level accountability tests begin As state level accountability tests begin to transition from paper administration to transition from paper administration to computer administration, several to computer administration, several issues arise: (Russell & Plati, 2002)issues arise: (Russell & Plati, 2002)

Issue #1Issue #1. Until all students are accustomed . Until all students are accustomed to writing on computers, a better to writing on computers, a better understanding of the extent to which the understanding of the extent to which the mode of administration affects student mode of administration affects student performance at different grade levels must performance at different grade levels must be developedbe developed

TransitioningTransitioning As state level accountability tests begin As state level accountability tests begin

to transition from paper administration to transition from paper administration to computer administration, several to computer administration, several issues arise: (Russell & Plati, 2002)issues arise: (Russell & Plati, 2002)

Issue #2Issue #2. Given the many computing . Given the many computing devices available, the effect of performing devices available, the effect of performing open ended items on desktop computers open ended items on desktop computers needs to be contrasted with performance needs to be contrasted with performance on cheaper and more portable writing on cheaper and more portable writing devices such as eMates and AlphaSmarts.devices such as eMates and AlphaSmarts.


As state level accountability tests As state level accountability tests begin to transition from paper begin to transition from paper administration to computer administration to computer administration, several issues arise: administration, several issues arise: (Russell & Plati, 2002)(Russell & Plati, 2002)

Issue #3Issue #3. Before testing programs offer . Before testing programs offer students the option of performing tests students the option of performing tests on paper or on computer, the extent to on paper or on computer, the extent to which handwritten versus computer which handwritten versus computer printed responses influence raters’ printed responses influence raters’ scores needs to be exploredscores needs to be explored


As state level accountability tests As state level accountability tests begin to transition from paper begin to transition from paper administration to computer administration to computer administration, several issues arise: administration, several issues arise: (Russell & Plati, 2002)(Russell & Plati, 2002)

Issue #4Issue #4. Administrative procedures for . Administrative procedures for administering tests on computers in administering tests on computers in schools must be developedschools must be developed

Ways to Bridge the GapWays to Bridge the Gap

Schools could decrease the Schools could decrease the amount of time students spend amount of time students spend working on computers so they do working on computers so they do not become accustomed to not become accustomed to writing on computerswriting on computers Not likely to happenNot likely to happen


States could eliminate paper and States could eliminate paper and pencil testing and have students pencil testing and have students perform open ended tests on perform open ended tests on computerscomputers Not likely to happen soonNot likely to happen soon Could place some students at a Could place some students at a

disadvantagedisadvantage


Administrators of tests could Administrators of tests could offer students the option of offer students the option of performing open ended tests on performing open ended tests on paper or on computerpaper or on computer Students would need to be able to Students would need to be able to

recognize their own strengthsrecognize their own strengths Would require additional efforts Would require additional efforts

from those administering and from those administering and scoring the testsscoring the tests


Those interpreting scores should Those interpreting scores should recognize the limitations of recognize the limitations of current testing programscurrent testing programs Probably the most reasonable Probably the most reasonable

solution in the short termsolution in the short term Does not “solve” the problemDoes not “solve” the problem Great emphasis is being placed on Great emphasis is being placed on

these tests by those who do not these tests by those who do not understand themunderstand them

ReferencesReferences

McKenzie, J. (1991, June) McKenzie, J. (1991, June) From Now From Now On, The educational Technology On, The educational Technology journal journal [On-line] 1 (6) Retrieved April [On-line] 1 (6) Retrieved April 10 from 10 from http://www.fno.org/fnojun91.htmlhttp://www.fno.org/fnojun91.html

References References (continued)(continued)

Russell, M. (1999, June 8) Testing on Russell, M. (1999, June 8) Testing on computers: A follow-up study computers: A follow-up study comparing performance on computer comparing performance on computer and on paper. and on paper. Education Policy Education Policy Analysis Archives Analysis Archives [On-line] 7 (20) [On-line] 7 (20) Retrieved April 11, 2003 from Retrieved April 11, 2003 from http://epaa.asu.edu/epaa/v7n20http://epaa.asu.edu/epaa/v7n20


Russell, M., & Haney, W., (1997, Russell, M., & Haney, W., (1997, January 15) Testing writing on January 15) Testing writing on computers: An experiment computers: An experiment comparing student performance on comparing student performance on tests conducted via computer and tests conducted via computer and via paper-and-pencil. via paper-and-pencil. Education Education Policy Analysis Archives Policy Analysis Archives [On-line], 5 [On-line], 5 (3) Retrieved April 11, 2003 from (3) Retrieved April 11, 2003 from http://epaa.asu.edu/epaa/v5n3.htmlhttp://epaa.asu.edu/epaa/v5n3.html


Russell, M., & Haney, W., (2000, March Russell, M., & Haney, W., (2000, March 28) Bridging the gap between testing 28) Bridging the gap between testing and technology in schools. and technology in schools. Education Education Policy Analysis Archives Policy Analysis Archives [On-line], 8 [On-line], 8 (19) Retrieved April 10, 2003 from (19) Retrieved April 10, 2003 from http://epaa.asu.edu/v8n19.htmlhttp://epaa.asu.edu/v8n19.html


Russell, M., & Plati, T. (2001) Effects of Russell, M., & Plati, T. (2001) Effects of computer versus paper computer versus paper administration of a state-mandated administration of a state-mandated writing assessment. [On-line] writing assessment. [On-line] Retrieved April 9, 2003, from Retrieved April 9, 2003, from http://ericsys.uncg.edu/TCRecordhttp://ericsys.uncg.edu/TCRecord%20-%20Writing%20-%20Writing%20Assessment.htm%20Assessment.htm


Russell, M., & Plati, T. (2002, May 29) Russell, M., & Plati, T. (2002, May 29) Does it matter with what I write? Does it matter with what I write? Comparing performance on paper, Comparing performance on paper, computer and portable writing computer and portable writing devices. devices. Current Issues in Education Current Issues in Education [On-line], 5 (4) Retrieved April 10, [On-line], 5 (4) Retrieved April 10, 2003 from 2003 from http://cie.ed.asu.edu/volume5/numbehttp://cie.ed.asu.edu/volume5/number4/index.html r4/index.html


(2002, September) STARS, Update#10, (2002, September) STARS, Update#10, Statewide writing assessment for Statewide writing assessment for 2002-2003 [On-line] Retrieved April 2002-2003 [On-line] Retrieved April 12, 2003 from 12, 2003 from http://www.nde.state.ne.us/stars/pdf/http://www.nde.state.ne.us/stars/pdf/update10.pdfupdate10.pdf

should students be allowed to use a word processor for the state writing assessment? pte 893 –...

Documents

writing programs

writing assessments

quality of student writing

state writing assessment

presence of computers

state testing programs

actual abilities of

scores reasons