document resume ed 412 278 author bond, linda; roeber ... · document resume. ed 412 278 tm 027...

DOCUMENT RESUME

ED 412 278 TM 027 734

AUTHOR Bond, Linda; Roeber, Edward; Braskamp, DavidTITLE Trends in State Student Assessment Programs: Fall 1996 Data

on Statewide Student Assessment Programs.INSTITUTION Council of Chief State School Officers, Washington, DC.;

North Central Regional Educational Lab., Oak Brook, IL.SPONS AGENCY Department of Education, Washington, DC.ISBN ISBN-1-884037-34-8PUB DATE 1997-00-00NOTE 42p.; For the annual survey, see TM 027 739.CONTRACT R279A50006-96APUB TYPE Reports Evaluative (142)EDRS PRICE MF01/PCO2 Plus Postage.DESCRIPTORS Data Collection; Disabilities; Educational Assessment;

*Educational Trends; Elementary Secondary Education; LimitedEnglish Speaking; Program Descriptions; Program Development;*Program Implementation; *State Programs; State Surveys;Test Construction; *Test Use; *Testing Programs; TrendAnalysis

IDENTIFIERS Council of Chief State School Officers

ABSTRACTThis document, fourth in a series, describes trends in

statewide assessment programs. It is based on surveys conducted in the pastf= the Acsociation of State Assessment Programs. States were asked todescribe the assessment programs they operated during the 1995-96 schoolyear. Part On of the survey asks each state to describe its existing

.1-sc(211_aborat,ive partners, and what it is developing. Part Two ofthe survey asks each state to describe its efforts in nontraditionalassessment. Part Three of the survey asks each state to describe eachassessment program, component, or groups of assessments that are used togather a set of data used for the same assessment purposes. For eachcomponent, states explain who is tested, what subjects are tested, and whattypes of assessments are used. In addition, states describe accommodationsprovided to English language learners and students with disabilities. Stateshave designed very different assessment systems, from use of anorm-referenced test alone to use of performance assessments. Most states,however, use a combination of multiple choice, short-answer,extended-response questions, performance tasks, or portfolios. This is thelast year the North Central Regional Educational Laboratory will participatein the collection of survey information on assessments; the program willcontinue under the direction of the Council of Chief State School Officers.(Contains 2 figures, 24 charts, and 8 references.) (SLD)

********************************************************************************

Reproductions supplied by EDRS are the best that can be madefrom the original document.

********************************************************************************

TRENDSin

StateStudent

AssessmentPrograms

FALL 1996Data on Statewide Student Assessment Programs

Linda Bond, CTB/McGraw-HillEdward Roeber, CCSSODavid Braskamp, NCREL

Council Of Chief State School OfficersOne Massachusetts Avenue, NW, Suite 700Washington, DC 20001-1431

BEST COPY AVAILABLE2

U.S. DEPARTMENT OF EDUCATIONOffice of Educational Research and Improvement

ED ED CATIONAL RESOURCES INFORMATIONCENTER (ERIC)

This document has been reproduced asreceived from the person or organizationoriginating it.Minor changes have been made toimprove reproduction quality.

Points of view or opinions stated in thisdocument do not necessarily representofficial OERI position or policy.

The Council of Chief State School Officers

The Council of Chief State School Officers (CCSSO) is a nationwide, nonprofit organization composedof the public officials who head departments of elementary and secondary education in the states, theDistrict of Columbia, the Department of Defense Education Activity, and five extra-state jurisdictions.CCSSO seeks its members' consensus on major educational issues and expresses their view to civic andprofessional organizations, federal agencies, Congress, and the public. Through its structure of standingand special committees, the Council responds to a broad range of concerns about education and providesleadership on major education issues.

Because the Council represents the chief education administrators, it has access to the educational andgovernmental establishment in each state and to the national influence that accompanies this uniqueposition. CCSSO forms coalitions with many other education organizations and is able to provideleadership for a variety of policy concerns that affect elementary and secondary education. Thus,CCSSO members are able to act cooperatively on matters vital to the education of America's youngpeople.

The State Education Assessment Center provides a central clearinghouse to improve data acquisition,monitoring, and the assessment of education. More recently, the State Collaborative on Assessment andStudent Standards (SCASS) was formed to network states and other groups to develop assessmentdesigns, as well as prototype and complete assessment components for a variety of content areas.Projects are taking place in arts education, comprehensive assessment design for Goals 2000/ESEA TitleI, health education, interdisciplinary standards and assessment, limited-English proficientassessment, literacy performance assessment, primary-level assessment, reading/writing performanceassessment, science education, social studies, assessing special education students, technical guidelinesfor performance assessment, and workplace readiness. The goal in all of these projects is to encouragethe development of higher quality student assessments at lower cost to the states.

The Council also supports the Association of State Assessment Programs (ASAP), an informal networkof state assessment staff, in order to provide direct assistance to the states.

COUNCIL OF CHIEF STATE SCHOOL OFFICERSGordon Ambach, Executive Director

STATE EDUCATION ASSESSMENT CENTERWayne Martin, Director

STUDENT ASSESSMENT PROGRAMSEdward Roeber, Director

One Massachusetts Avenue, N.W., Suite 700Washington, D.C. 20001 - 1431

(202) 408-5505

Copyright 1997 by the Council of Chief State School Officers, Washington, D.C.All rights reserved with the exception of reproduction for educational purposes.

ISBN # 1-884037-34-8

3

TRENDSin

State.Student

AssessmentPrograms

FALL 1996Data on Statewide Student Assessment Programs

Linda Bond, CTB/McGraw-HillEdward Roeber, CCSSODavid Braskamp, NCREL

Prepared By the North Central Regional Educational Laboratory for submission under contract with theCouncil of Chief State Schools Officers. The preparation of this document was financed by funds providedby the U.S. Department of Education award 413279A50006-96A and completed April 24, 1997.

Council Of Chief State School OfficersOne Massachusetts Avenue, NW, Suite 700Washington, DC 20001-1431

TABLE OF CONTENTS

CHAPTER ONE

Introduction to the State Student Assessment Programs Database 1

CHAPTER TWO

Overview of State Student Assessment Programs 3Pervasiveness of Statewide Assessment 3

Chart 2-1: Which Students are Assessed 3Which Grades are Assessed 3

Chart 2-2: Grades Assessed 3Why States Assess with Statewide Assessments 3

Chart 2-3: Types of Assessment Purposes 4Chart 2-4: Most Reported Assessment Purposes 4Figure 2-1: High School Graduation Testing 4

When Assessments are Administered 4Chart 2-5: When Students are Assessed 5

What Subjects are Assessed 6Chart 2-6: Subjects Assessed 6

How States Assess Students 6Chart 2-7: Types of Assessments Used 6Chart 2-3: Types of Test Items Administered 6Chart 2-9: Commercial Publishing Involvement 7Chart 2-10: Subjects Assessed by Test Type 7

Summary 7

CHAPTER THREE

New Forms of Statewide Assessment 9An Overview of Non-Traditional Assessment in the United States 9

Chart 3-1: Types of Test Items Administered 9Chart 3-2: Major Subjects Assessed with Non-Traditional Items 9Chart 3-3: Non-Traditional Items in Use: Language Arts and Writing 10Chart 3-4: Non-Traditional Items in Use: Mathematics and Science 10

Nontraditional Exercise Development in the 1995-1996 School Year 10Chart 3-5: Development of Non-Traditional Items: Language Arts and Writing 10Chart 3-6: Development of Non-Traditional Items: Mathematics and Science 11

A Blend of Assessment Approaches is Most Common 11Figure 3-1: Test Type Combinations 11

The Jury is Still Out 11Why a Blended Assessment Approach 13Constraints on Developing Nontraditional Assessments 14Summary 15

CHAPTER FOUR

Special Topics: Assessment of Students with Disabilities and Limited EnglishProficient Students and Title I Assessment and Evaluation Plans 17

Part I: Assessment of Students with Disabilities and Limited English Proficient Students 17Overall Policy for Ensuring the Participation of Students with Students with Disabilities

and English Language Learners in the State Assessment Program 17Participation of Students with Disabilities and Limited English Proficient Students in

Statewide Assessment 18Survey Results Concerning the Inclusion and Exclusion of Students with Disabilities and

English Language Learners 19Chart 4-1: Students with Disabilities: Exemptions and Accommodations Policy 19Chart 4-2: English Language Learners: Exemptions and Accommodations Policy 19

Accommodations Allowed 19Types of Accommodations Allowed 20

Chart 4-3: Accommodations Allowed 20Public Reporting of Assessment Results 20

Chart 4-4: Public Reporting Policy for Students with Disabilities and EnglishLanguage Learners 21

What Next 21Part II: State Title I Assessment and Evaluation Plans 22Content and Performance Standards 22

Chart 4-5: State Goals, Standards and Frameworks 22Chart 4-6: Performance Standards Setting 22

Adequate Yearly Progress 22Chart 4-7: Definition of Adequate Yearly Progress for Title I Purposes Status 23

Status of State Title I Assessment Plans 23Chart 4-8: Status of State Transitional and Final Title 1 Assessment Plans 23

Summary 23

CHAPTER FIVE

Statewide Assessment History and Trends 25Introduction 25Criterion-Referenced Assessment and Minimum Competency Tests 25Advent of Writing Assessment 25Expansion to Other Subject Areas 26Performance Assessment 26Professional Development on Assessment 27Norm-Referenced Tests 27National Efforts at Joint Development 29Accommodating Students with Disabilities 29Assessing English-Language Learning Students 30Future Issues and Their Impact on State Assessment 30

RESOURCES 33

SUMMARY TABLE 35

6

LIST OF CHARTS

Chart 2-1: Which Students are Assessed 3

Chart 2-2: Grades Assessed 5

Chart 2-3: Types of Assessment Purposes 4

Chart 2-4: Most Reported Assessment Purposes 4

Chart 2-5: When Students are Assessed 5

Chart 2-6: Subjects Assessed 6

Chart 2-7: Types of Assessments Used 6Chart 2-8: Types of Test Items Administered 6

Chart 2-9: Commercial Publishing Involvement 7Chart 2-10: Subjects Assessed by Test Type 7Chart 3-1: Types of Test Items Administered 9

Chart 3-2: Major Subjects Assessed with Non-Traditional Items 9

Chart 3-3: Non-Traditional Items in Use: Language Arts and Writing 10Chart 3-4: Non-Traditional Items in Use: Mathematics and Science 10Chart 3-5: Development of Non-Traditional Items: Language Arts and Writing 10Chart 3-6: Development of Non-Traditional Items: Mathematics and Science 11Chart 4-1: Students with Disabilities: Exemptions and Accommodations Policy 19Chart 4-2: English Language Learners: Exemptions and Accommodations Policy 19Chart 4-3: Accommodations Allowed 20Chart 4-4: Public Reporting Policy for Students with Disabilities and

English Language Learners 21Chart 4-5: State Goals, Standards and Frameworks 22Chart 4-6: Performance Standards Setting 22Chart 4-7: Definition of Adequate Yearly Progress for Title I Purposes Status 23Chart 4-8: Status of State Transitional and Final Title I Assessment Plans 23

LIST OF FIGURES

Figure 2-1: High School Graduation Testing 4Figure 3-1: Test Type Combinations 11

PREFACE

This is the fourth edition of the Trends in State Student Assessment Programs Fall 1996, a product of thecollaboration between the Council of Chief State School Officers (CCSSO) and the North CentralRegional Educational Laboratory (NCREL). This is the final year that NCREL will participate in thecollection and dissemination of the results from the annual survey, since Linda Bond (formerly withNCREL) has taken a position with CTB/McGraw-Hill. The survey will continue in the future underthe direction of CCSSO, with new partners, since the annual survey remains the single best source forinformation about statewide student assessment programs, and its contents are comprehensive, accurate,and up-to-date.

This document describes the trends in statewide assessment programs. It is based on surveys conducted inthe past for the Association of State Assessment Programs (ASAP) by its chair, Edward Roeber. Thisyear's survey was extensively revised by an advisory committee put together by CCSSO and NCREL,and the surveys were mailed to states in September 1996. States were asked to describe theirassessment program(s) they operated during the 1995-96 school year. Surveys were received fromOctober 1996 through January 1997 they were processed first by CCSSO and then were sent to NCREL fordata entry, survey completion, and editing. Between December 1996 and February 1997, each assessmentdirector received a copy of his or her state's information for editorial review and updates. Thesechanges were made by NCREL. NCREL produced the final tables, tabulations of results, and all chartsand graphs used to display the results.

The data from 1995-96, plus the prior four years, provide a rich lode of information on the status of andtrends in state assessment practice. This report provides the reader with information on the currentstatus of programs, as well as descriptions of how the programs have changed over the years. Acompanion document, Status of Statewide Student Assessment Programs Fall 1996, is also availablefrom CCSSO. The data are also available in electronic form (as Acrobat files suitable for Macintoshand Windows). An order form is attached to this document. Selected and updated information from thedatabase and these documents are available at CCSSO's and NCREL's World Wide Web sites(http: / /www.ccsso.org or http: / /www.ncrel.org).

Please feel free to contact us if you have any questions about this document, the assessment programs instates, or other questions.

David Braskamp Ed RoeberLinda Bond CCSSONCREL

Chapter OneIntroduction to the State Student Assessment Programs Database

The topic of student assessment generatesconsiderable controversy among educatorsand members of the public. Some view large-scale assessment programs as criticalelements of the reform and change needed inAmerican schools. Two primary reasons forthis are: (1) assessment can providedirection and motivation to students, parents,teachers and others to help students learn theskills needed to succeed both in school and inlife after school; and (2) assessment programscan help gauge the success of our schools.An indication of the strength of their appealis the number of states that currently haveassessment programs: 46. Of the remainingfour states, Colorado and Minnesota are atwork developing assessment programs.Iowa and Nebraska are the only two statesthat are not presently administering ordeveloping a statewide assessment program.

However, there are educators and membersof the public who view many large-scaleassessments with reservations. Critics feelsuch programs can exert negative pressureon teachers and students. Much of the debatesurrounds such issues as the content coveredby the assessments, the type of assessmentused, how the assessments are scored and theuses made of the assessment results.However viewed, large-scale statewideassessment programs are a fact of life in theUnited States.

While state assessment programs share somecommon purposes and methods, they canalso be quite different. Differences exist forvarious reasons - for example, theeducational policy climate in the state, thetechnical quality issues surrounding the useof assessment to make high-stakes decisions,or the status of curricular reform in the state.We need to recognize these differences inorder to understand the assessment

1

programs that exist and the options that areavailable to change these programs.

In addition, we need to recognize themovement in Washington, DC to limit thefederal role in education by shifting moreresponsibility for education to the states. Aresult of this shift has been that states havebeen given more control over the educationalresources provided to their schools.Similarly, states have shifted moreresponsibility and control from the state tothe district and school levels. The price forincreased flexibility and control hastraditionally been increased accountabilityand, therefore, increased assessment.Historically, much assessment activity andexperimentation in new forms of assessmenthave occurred in states. We will be keepingan eye on how these shifts in responsibilitywill affect state assessment and whether stateassessment will continue to play a major rolein educational reform.

The Association of State AssessmentPrograms (ASAP), an informal organizationof state assessment directors, began collectinginformation about large-scale assessmentprograms at the state level in 1977. Theresults of the annual ASAP surveys wereprovided to states in the form of a writtensummary of each state's assessment program.In 1991, Ed Roeber, ASAP's chair, becamedirector of student assessment programs forthe Council of Chief State School Officers(CCSSO). A partnership with the NorthCentral Regional Educational Laboratory(NCREL) led to the current form of the StateStudent Assessment Program (SSAP)database. This report is a result of the fifthyear of that partnership. However, this is thefinal year for this partnership, as Linda Bondhas left NCREL. CCSSO will be continuingthe survey in the future with new partners.

As the amount of information increases overtime, we are able to provide more meaningfulinformation to states because we are able tomonitor patterns of change in stateassessment programs. As data collectioncontinues in the future, we hope to sharpenthe analysis of change in statewideassessment practices.

The survey annually collects three kinds ofinformation. Part One of the survey askseach state to describe its existing program, itscollaborative partners and what it isdeveloping. Part Two of the survey asks eachstate to describe its efforts in nontraditionalassessment. Part Three of the survey askseach state to describe each assessmentprogram, component, or groups ofassessments that are used to gather a set ofdata used for the same assessment purposes.For each component, states explain who istested, what subjects are tested and whattypes of assessments are used. In addition,states describe accommodations provided tostudents with disabilities and Englishlanguage learners. From this detail, we canbuild a more detailed picture of whatstatewide assessment programs look like andhow they are attempting to accomplish theirstate assessment goals. This report is asummary to provide an understanding ofwhat the 50 states are doing and how theyare doing it.

10

2

Chapter TwoOverview of State Student Assessment Programs

This chapter provides an overview of theassessments that states conduct. A tabularoverview appears in the Summary Table inthe Appendix. The detailed responses foreach state to the survey are available in acompanion publication, Annual Survey of StateStudent Assessment Programs, Fall 1996.

Pervasiveness of Statewide Assessment

States have the option of administering theirassessments in different ways: to all eligiblestudents, to a sample of students, or tostudents on a voluntary basis. As indicatedin Chart 2-1, most states choose to include allpossible students on at least one set ofassessments.

Chart 2-1Which Students are Assessed

One or more assessmentsgiven to all students

No mandatedstate assessments

All assessments given toa sample of students

All assessmentsare voluntary

o 10 20 30 40Number of States

50

Given that 46 states have administeredstatewide student assessments in the 1995-96school year, statewide student assessment isan important factor in making educationaldecisions in states nationwide.

Which Grades are Assessed

States assess students in kindergartenthrough grade 12, but, as in indicated inChart 2-2, some grades are assessed morethan others.

KindergartenOneTwo

ThreeFourFiveSix

SevenEightNineTen

ElevenTwelve

0

Chart 2-2Grades Assessed

10 20 30 40Number of States

50

Elementary school grades are assessed moreconsistently than high school grades. Inaddition, few states do not assess earlychildhood grades (kindergarten throughsecond grade). The three grades that areassessed the most often are grades 4, 8 and11. These assessments occur at the end ofelementary, middle and high school andcommonly serve as summative assessmentsof elementary or high school performance.

Why States Assess with StatewideAssessments

There are two basic reasons why statesconduct assessments: to improve instructionand to hold schools and studentsaccountable. Charts 2-3 and 2-4 (next page)summarize the basic purposes and highlightthe main purposes.

3 n

Chart 2-3Types of Assessment Purposes

Instructional Purposes

School Accountability

Student Accountability

Staff Accountability

0 10 20 30 40 50Number of States

Chart 2-4Most Reported Assessment Purposes

Improvement of Instruction

Program Evaluation

Curriculum Planning

School Performance Reporting

Student Diagnosis

High School Exit Exam


As Chart 2-3 indicates, virtually all stateswith assessment programs report using themfor improving instruction. Purposesdesigned to impact classroom or building-level practice (e.g., curriculum planning) aremore likely to be reported than thoseaffecting individual students (e.g., studentplacement). In a similar way, school-levelaccountability purposes (e.g., schoolperformance reporting) are more reportedthan purposes that impact students directly(e.g., high school graduation testing).Statewide student assessment programs arebasically not used for staff accountabilitypurposes.

High school graduation tests are the mostpopular type of individual studentaccountability purpose reported. States may

124

use these tests as an high school exitrequirement, to grant an endorsement on adiploma, or to receive an honors diploma.Figure 2-1 shows the distribution of highschool graduation tests of all typesnationwide.

Figure 2-1High School Graduation Testing

0ITP

Graduation Exams Used

El No Statewide Assessment Endorsed Diploma Only

Em High School Exit Exam. Endorsedr-i No Graduation ExamNM Diploma. and Honors Diploma

High School Exit Exam Only M High School Exit Examand Endorsed Diploma

The use of these tests is concentrated in twoareas: almost all of the states on the southernUnited States border and most of the easternseaboard. The midwest and far west UnitedStates generally do not use high schoolgraduation tests.

When Assessments are Administered

Assessments meant to. serve the purposes ofimprovement of instruction and educationalaccountability would ideally be administeredat different times of the school year.Assessments used to improve instructionwould be given in the early part of the schoolyear in order to provide timely feedback forschool administrators and teachers.Assessments used for educationalaccountability would be given at the end ofthe school year to measure studentknowledge at the latest possible time. Aschart 2-5 indicates, almost all states assesssome or all students near the end of the

school year (March, April, or May), andmany states also administer someassessments during the fall semester(September to December).

Chart 2-5When Students are Assessed

March to May

September toDecember

January orFebruary

Summer

0 10 20 30 40

Number of States

50

The many assessments administered in thelate spring may be ideal for accountabilitypurposes, but they are almost alwaysreportedly used in some way to improveinstruction. When assessments areadministered at the end of the school yearthere is virtually no good way for states tohelp improve the instruction or curriculumfor students who took those tests.

Issues around designing assessments that canmeet the purposes of instructionalimprovement and accountabilitysimultaneously are also important toconsider. For example, two thirds of stateswith assessment programs report using thesame assessment for instructionalimprovement and school performancereporting. Combining the purposes ofinstructructional improvement and schoolaccountability in one program may requireconflicting differences in test design andavailability of the results.

Designing an assessment program to meethigh-stakes accountability purposes typicallyrequires standardization of test content,administration and scoring. Accuracy ofscoring and standardization of test

5

administration procedures are paramount.Test security is high, with results determinedat a centralized scoring center and returnedweeks, sometimes months, after theassessment is administered.

The very safeguards that ensurecomparability and fairness limit the utility ofthe results for instructional decision making.For an assessment to be effective as aninstructional improvement tool, the resultsneed to be made available almostimmediately so that teachers can adjust theirinstruction. Reviewing assessments over thesummer may be helpful for curriculumplanning, but teachers need access to ongoingassessment information to modifyinstructional strategies within the classroom.A classroom-based assessment system, albeitsomewhat standardized by virtue of thelearning goals being assessed, requirescontinuous, unobtrusive collection ofassessment data, flexible administration andimmediate feedback. Unfortunately, thisflexibility is typically seen to violate thestandardization necessary for accountabilitypurposes.

The state assessment directors acknowledgethe difficulty inherent in using the sameassessment for both instructionalimprovement and accountability purposes.However, state law and regulation oftenrequire them to do so. States, therefore, aredesigning assessment systems that try tocapture both sets of purposes in ways tominimize the conflict between them.

13

What Subjects are Assessed

As Chart 2-6 indicates, traditional subjects arethe ones most assessed in statewideassessments.

Mathematics

Language Arts

Writing

Science

Social Studies

Chart 2-6Subjects Assessed

0 10 20 30 40

Number of States

50

Mathematics and language arts are assessedby almost all states. Writing, science andsocial studies are assessed by most states.As it has been for many years, the three R's(reading, writing, and arithmetic) remain themost commonly assessed subjects.

How States Assess Students

As indicated in Chart 2-7 and Chart 2-8,states use a variety of tests and item types toassess student knowledge.

Chart 2-7Types of Assessments Used

Writing Assessment

Criterion-referenced Test

Norm-referenced Test

Performance Assessment

Portfolios


146

Chart 2-8Types of Test Items Administered

Multiple-choice Items

Extended Written Response

Short Answer

Examples of Student Work

Performance Tasks

Projects


Writing assessment and criterion- and norm-referenced tests are the most common typesof assessments reported by states.Performance assessment by states continuesto rise (up from 17 last year), while portfolioassessment is still used by only a few states.

For the first time, we asked states to reportspecifically what types of items are used onthe assessments they administer statewide.Almost all states use multiple-choice items,traditionally found on criterion- and norm-referenced tests, because of the psychometricadvantages, ease of administration and lowcost associated with them. Extended writtenresponse, commonly found on writingassessments, requires students to write inresponse to writing prompts. Surprisingly,examples of student work from the classroomare used by more states than just those usingportfolios. The number of states usingperformance tasks and projects is less thanthe number of states using performanceassessment. This indicates that the term"performance assessment" still can indicate avariety of assessment types, includingextended written response, performancetasks, and/or projects. In many cases, statesreporting the use of performance assessmentprobably were pointing to their use ofextended written response items.

In designing, administering and scoringstatewide assessments, states often dependon commercial publishing companies to helpwith some or all of the work. As indicated inChart 2-9, commercial test publishers haveworked with most states on at least onestatewide assessment.

Chart 2-9Commercial Publishing Involvement

Commercial TestPublisher Involvement

Service or Test Provided

Off -the-Shelf Test

Custom Developed

Customized Off-the-Shelf

Commercial Item Bank Items


Off-the-shelf tests (e.g., Terra Nova) are usedby half of the states with programs interestedin comparing groups of students to normgroups, generally national or state. Thecustom-developed tests are a joint venturebetween the states and commercial testingpublishers to design a test that matches thestate curriculum more closely than off-the-shelf tests.

Chart 2-10 displays the minimum number ofstudents assessed by each type of assessment.

Chart 2-10Subjects Assessed by Test Type

Criterion-referenced Test

Writing Assessment

Norm-referenced Test


Portfolios

0 3 6 9 12 15

Number of Students (Millions)

7

States have a range of the number ofstudents, may use a sample of students ormake certain assessments voluntary, givingus a view of how students are assessednationwide. The multiple-choice test is themost dominant assessment type usedstatewide. States appear to be almost ascomfortable with administering and scoringwriting assessments with extended responsesas they are with administering multiple-choice tests. Performance assessments, withrelatively fewer students for the number ofstates reporting them, are often voluntary orused in states with smaller studentpopulations. They are more difficult to useon a large-scale level than multiple-choice orwriting assessments.

Summary

Statewide assessment is an important factorin making instructional improvement andaccountability decisions nationwide. Inresponse to these potentially conflictingneeds, states have designed very differentprograms from only using a norm-referencedtest, to using only performance assessments.Most states, however, use a combination ofmultiple-choice, short answer, extendedresponse, performance tasks, or portfolios.As states continue to increase the number ofdifferent combinations, it will be interestingto watch how they grapple with the pressureto use tests to improve instruction and holdschools, students and staff accountable withinthe same or different assessment programs.

15

Chapter ThreeNew Forms of Statewide Assessment

Conventional multiple-choice assessmentscontinue to be the most popular form ofassessment in state assessment programs, butthe number of states that rely exclusively onmultiple-choice assessment is small. Thisnumber has changed slightly from a low of 5states in 1993-94 to a high of 9 states in 1994-95.7 states reported using only multiple choiceitems in their state assessment programs duringthe 1995-96 school year, the most current yearfor which data is available. Although exclusivereliance on multiple-choice is uncommon, forty-one states report using multiple-choice items aspart of their state assessment program, andtwenty-nine of these have at least oneassessment component within their system thatrelies exclusively on norm-referenced and/orcriterion-referenced multiple-choice items.Clearly, multiple-choice remains a favoriteassessment choice for states, but it is not theonly choice.

Prompted by a growing concern that the kindsof skills needed for success in. the 21st centurygo beyond those that are typically taught andassessed in traditional educational settings,states have been revising their student learninggoals, their curricula and the forms ofassessment they use to measure mastery ofthose student goals. As a major part of thiseducational reform effort, states have exploredalternative' forms of assessment, which requirestudents to produce answers rather than simplyselect correct answers.

An Overview of Non-TraditionalAssessment in the United States

This year, we tried to get a better idea of whattypes of assessment items are considered"alternative" or "performance" assessment by

1Throughout this chapter, alternative assessment and non-traditional assessment refer to non-multiple-choice assessment.The two terms are used interchangeably.

states. Chart 3-1 displays the types of testitems administered.

Chart 3-1Types of Test Items Administered

Multiple-choice Items

Extended Written Response

Short Answer

Examples of Student Work

Performance Tasks

Projects


The most common forms of "alternative"assessment are assessments typically in writing,but also in mathematics, language arts, scienceand social studies. The more "hands-on"forms of alternative assessment performancetasks, portfolios and student projects simplyare not that common.

As Chart 3-2 indicates, subjects assessed withnon-traditional items are the same as thoseassessed with traditional items.

Chart 3-2Major Subjects Assessed with

Non-Traditional Items

Writing

Mathematics

Language Arts

Science

Social Studies

0 10 20 30 40Number of States

50

916

Writing continues to be the subject mostcommonly assessed with alternativeassessment.

This is not surprising, considering that writingis one skill that most agree cannot be assessedexcept by having the student write.

The types of non-traditional items in use inthe subjects of language arts and writing aredetailed in Chart 3-3. Extended response isclearly the favorite choice for assessingwriting, while short answer and extendedresponse items are used most often forlanguage arts.

Chart 3-3Nontraditional Items in Use:Language Arts and Writing

Multiple-choice withMultiple Correct Answer

Multiple-choice, withStudent Explanation

Short Answer

Extended Response

Hands-on PerformanceAssessment

Portfolio orLearning Record


1,11E9Writing OLanguage Arts I

With mathematics and science, short answerand extended response are again the mostpopular forms of non-traditional items statesuse within their assessment programs.

Considering the need to ensurestandardization of format, presentation,administration and scoring of stateassessments, especially where student orschool scores will be compared to a uniformstandard, using these fairly traditionalapproaches to non-traditional assessmentmakes sense. Standardizing more "hands-on" assessments, such as laboratoryexperiments or student projects, is moredifficult. It is possible to score short answer

10

items as right or wrong, and states have hada long history of using extended responseitems with the assessment of writing since the1970s.

Chart 3-4Nontraditional Items in Use:Mathematics and Science

Multiple-choice withMultiple Correct Answer

Multiple-choice, withStudent Explanation

Short Answer

Extended Response

Hands-on PerformanceAssessment

Portfolio orLearning Record


IlEtiMathematics CIScience I

Non-Traditional Exercise Development in the1995-1996 School Year

There is considerable development of non-traditional test items taking place withinstates, with the vast majority of the test itemsbeing piloted, ready for use or in use. (SeeChart 3-5 for language arts and writing;Chart 3-6 for mathematics and science.)

Chart 3-5Development of Non-Traditional Items:

Language Arts and Writing

PredevelopmentStages

Development Stages

Piloting, Ready for Use,or In Use

0 5 10 15 20 25 30Number of States

I 1211Writing OLanguage Arts

Chart 3-6Development of Non-Traditional Items:

Mathematics and Science

PredevelopmentStages

Development Stages

Piloting, Ready for Use,or In Use

0 5 10 15 20 25 30Number of States

ei Mathematics ID Science]

This pattern may indicate that most of thestates are very close to implementing or havealready implemented the alternatiVeassessment items they plan to develop.These items tend to be short answer or essay(extended response) items and in thetraditional subject areas of writing, languagearts, mathematics, science, and social studies.Wyoming is a notable exception; it assessesonly vocational education students with awide variety of alternative assessmentapproaches.

A Blend of Assessment Approaches is MostCommon

Most states have added these alternative itemsto their existing forms of assessment, but somehave created separate non- traditionalassessments as part or their entire stateassessment program. In fact, twenty-threestates report having at least one assessmentcomponent that includes no multiple-choiceitems.

In looking at Figure 3-1, we see that most stateshave some combination of assessment types aspart of their state assessment programs. Of theforty-six states with state assessment programs,7 states use just multiple-choice items, ten usesome form of extended written response inaddition to multiple-choice, 9 rely upon

11

TCOPYAVAILARLF

multiple-choice, short-answer and extendedwritten responses, fifteen rely upon multiple-choice, extended response and other forms ofalternative assessment, and 5 rely uponalternative forms of assessment only. These 5are Delaware, Kentucky, Maine, Vermont andWyoming. Delaware administers a writingassessment, Kentucky uses extended responseplus examples of students' work, Maineemploys extended response items, Vermontuses short answer and extended responseitems, and Wyoming2 relies upon observation,examples of students' work, and projects.

Figure 3-1Test Type Combinations

Combinations of

ElNo Statewide Assessment

Alternative Items. ExtendedResponse, and Multiple-Choice

I-1 Extended Response andMultiple-Choice

Item Types

Extended Response. ShortAnswer, and Multiple-Choice

Multiple- ChoiceChoice Only

1111 Alternative Items Only

The Jury is Still Out

While states have been moving toward theinclusion of alternative forms of assessment at asteady pace, few have made broad-sweepingchanges to their programs, and those that havemade changes found out too late that theirpublics were not as enthusiastic about thechanges as were the originators of theprograms. Three of the states that were farthestalong in their use of alternative assessments as aprimary assessment strategy (California,Kentucky and Arizona) have hit major detours

2 Wyoming assesses students in vocational educationonly.

l8

due to technical problems, cost and publiccriticism of content. In California, the state'smajor assessment program, the CaliforniaLearning and Assessment System (CLAS),which relied heavily upon performanceassessments and constructed-response items,has been discontinued. In its place will be astatewide basic and applied academic skillsassessment at key grade levels and a voluntaryPupil Incentive Testing Program. Thehighlights of the Pupil Incentive TestingProgram include:

1.) Districts will receive $5.00 per student toselect a published achievement test

2.) Districts must assess students in reading,spelling, written expression andmathematics by a standardized test from astate-approved test list

3.) Districts must administer the tests to alleligible students from grades 2 through 10.

4.) Districts must report the results annually totheir students, teachers, parents andgoverning board.

California continues, however, to usealternative forms of assessment in its Career-Technical Assessment Program and its GoldenState Exams (end-of-course exams), a fact that isnot well known.

Arizona is another state that had a major non-traditional assessment program suspended inschool year 1994-1995 and for now onlyadministers a norm-referenced, multiple-choicetest.

Another state that moved away from multiple-choice items toward the exclusive use ofperformance assessments and portfolios hasfaced similar problems. In Kentucky, multiple-choice items have been returned to theassessment program with the addition of atraditional, standardized test as well. In relyingon performance assessments and portfoliosexclusively, Kentucky found it needed moreinformation per student, and it needed to be

collected in a cost-effective and technicallysound manner.

Two other states that were moving toward aheavier reliance on performance assessmenthave had the funding for their programswithdrawn, but one of them has movedforward despite controversy.

In Wisconsin, the funding for a performance-assessment component of the Wisconsinassessment system was dropped after a three-year developmental period was nearcompletion. Full implementation in languagearts and mathematics had been planned fornext year.

Indiana similarly lost some of its funding afterdeveloping and piloting a new assessmentprogram, which included a move away fromnorm-referenced testing to criterion-referencedtesting, and the inclusion of a substantialnumber of open-ended items and performancetasks. The new legislation called for thecontinuation of the norm-referenced test with itscriterion-referenced supplement, one open-ended mathematics task and one writingsample at benchmark grade levels. Even thisminimal inclusion of alternative assessment waschallenged in a lawsuit claiming the testinvaded the privacy of children. Indiana wonthe suit.

Indiana was able to expand its use of open-ended tasks last year, at least in part becausenew legislation built into the system extensivepublic review of all rubrics and open-endeditems by a citizen's review panel. In addition,rubrics for the writing tasks are released to thepublic before the administration of theassessment, and all open-ended tasks arereleased after administration of the assessment.In this way, Indiana has been able to maintainwriting assessment tasks, integratedreading/writing tasks, and mathematics open-ended tasks in addition to a norm-referencedtest and a criterion-referenced supplement. Thestate was also willing to pay for the increased

12

is

costs incurred due to the release of theassessment materials, thus allowing theprogram to continue and to grow.

The first three states discussed (California,Kentucky, and Arizona) were among theleaders in the alternative assessmentmovement Kentucky is the only one of thethree that remains in the forefront; California'sprogram is defunct, and Arizona's is on hold.The other two states discussed, Wisconsin andIndiana, may have been caught in the flak thatresulted from the very public attacks against thefirst three states' programs. Political battles,concern over so-called "non-objective" and"intrusive" forms of assessment, high costs andtechnical difficulties seem to be at the heart ofmany of the concerns expressed aboutalternative assessment activity. Some of theseconcerns will be discussed more fully later inthis chapter.

Despite these roadblocks, the amount of stateactivity in the development and use ofalternative assessment items is considerable.The chart having to do with state developmentof non-multiple-choice items within the printedversion of this year's database covers 21 pages!While the majority of these projects areextended response and short-answer, there arequite a number of states that are developing orhave developed hands-on performance tasksincluding: Connecticut, Delaware, Idaho,Illinois, Kansas, Kentucky, Mississippi, NorthCarolina, New York, Vermont, West Virginiaand Wyoming. Projects, exhibitions anddemonstrations are also being developed inIdaho, Illinois, Kansas, Minnesota, New York,Ohio and Wyoming. Clearly the benefits of thisform of assessment are great enough for statesto work toward overcoming the barriers. Morediscussion on these barriers and ways toovercome the barriers follow later in thischapter.

Why a Blended Assessment Approach?

The fact that states are moving toward the useof multiple types of assessment makes sense.After all, no single form of assessment isappropriate for all purposes. There are trade-offs involved in the use of any assessmentstrategy.

Alternative forms of assessment are beingexplored for many reasons. First, there is anational movement to clearly define studentstandards; that is, what students should knowand be able to do. Along with the standardsmovement comes a desire to accurately describewhat students now know and can do vis a visthe standards. Alternative forms of assessmentare being designed to make thesedeterminations, particularly with standards thatcannot be assessed with a paper and pencil testIn addition, many of the standards are differentfrom what has traditionally been taught inschools. Changes in the workplace and in theskills needed for life in an information agesuggest that students need knowledge andskills that will enable them to solve increasinglycomplex problems. Some of these skills cannotbe assessed using traditional, multiple-choiceassessment, and this is causing many states toexplore alternatives.

Multiple-choice assessments require students toselect a "right" answer from among several"wrong" answers. These assessments are usefulfor assessing knowledge and thestraightforward application of that knowledge.On the other hand, open-ended assessmentsthat require students to generate their ownsolutions to assessment problems or tasks arebecoming increasingly necessary to assess newlearner outcomes that call for more complexapplications of knowledge and skill. Manystates are concerned that relying exclusively ontraditional multiple-choice, basic skillsassessments results in a narrowed curriculumthat produces students who memorize a lot offacts and skills, but have little ability to applythem to real-life situations. However, these

13

assessments are easy to administer, fairlyinexpensive and yield a broad sample ofstudent performance in a relatively short periodof time. They simply cannot be used to assessmore complex applications of studentknowledge, and they offer few clues to theteacher about why the student gave a correct orincorrect answer.

This is why states are adding alternative formsof assessment One of the major benefits ofnon-traditional assessment is that, in addition tojudging the correctness of the student's answer,the appropriateness of the procedure that thestudent employed is also considered. Thisgives teachers more information for diagnosticpurposes because the teacher can determinewhere the student is having difficulty. But non-traditional assessments also have their trade-offs most notably, the increased cost and timeassociated with their development,administration and scoring. Ensuring thereliability of these assessment results has alsoproven costly and difficult, although thebenefits in improved assessment of complexskills and the modeling of good instruction isworthwhile to many states. Another difficultyof non-traditional assessments isgeneralizability. Different performance tasksevoke different levels of skill from the samestudents. This limits the likelihood that a givenperformance on a small sample of tasks will bestrongly indicative of the student's overallability.

For these reasons, most states are combiningtraditional assessment programs with non-traditional assessments. They are alsoexamining their traditional programs, which aregetting a face-lift with new content andstandards.

Constraints on Developing Non-TraditionalAssessments

States continue to struggle with a number ofconstraints concerning the implementation ofalternative assessments. These include: time

and cost constraints, technical quality issues,and resistance to change.

Time. There are two time constraints. The firstis the time to develop a test This iscompounded by a sense of urgency:several states reported legislativemandates to put their programs intoplace before the tests were ready. Thesecond constraint is the time toadminister an alternative assessment inthe classroom. In the time it would takea student to complete one or twoperformance tasks, that same studentcould have completed 200 items on amultiple-choice test.

Cost Again, there are several issues. Sincethe technologies are new, theprocedures to develop items or tasks arenot nearly as well established as in thedevelopment of multiple-choiceassessments. It takes people more timeto develop and test such items. Thetime that testing requires in theclassroom also adds to the cost ofalternative assessment Alternativeassessment items are more expensive toscore than multiple-choice tests.Alternative assessments requireteachers or other professionals to recordobservational data or make judgmentsabout extended artifacts of studentperformance. This requires the skill andtime of individuals if the work of manystudents is to be assessed.

Professional development expense. There isalso a considerable expense for theprofessional development of alternativeassessment. Staff need to understandthe changes, need training in theconsistent conduct and use ofalternative assessment items, and needsupport in using and reporting theresults of alternative assessmentHowever, the professional developmentbenefits derived from teachers who

14 21

design, implement and/or score the non-traditional assessments is a benefit manystates cite as a major reason to continuethis work

Technical Quality. Because non-traditional itemsare a new technology, it is far from easy toobtain uniform results. While sometechnical concerns are not unique to non-traditional items and may in fact pose lessof a threat, i.e., the issue of validity (are weassessing important learning?), theyremain real, and others, such as reliability(student results are an accurate reflectionof the student's performance rather than aresult of extraneous influences such aswho does the scoring) or generalizability(scores on this assessment would besimilar to scores on similar assessments),continue to be daunting. There is somuch more flexibility with non-traditionalassessment that maintaining uniformity ofadministration, scoring and interpretationis more difficult

Public Engagement Resistance to new forms ofassessment comes mostly from students,teachers and parents. All three are morefamiliar with standardized tests whereminimal preparation and administrationtime are required, and reports are familiarand support a norm-referenced, gradingsystem (A,B,C,D,F). Organized parents'groups have fought the new assessmentsin a number of states due to concerns thatthe open-ended nature of performanceassessments will allow students to bejudged by the personal values theyinclude in their responses rather than bytheir academic performance.

Summary

In reviewing the data on non-traditionalassessment activity this year, it would appear thatwhere states have implemented performanceassessment as a slow and deliberate processwithout much fanfare, their programs have beenspared. Connecticut was one of the first states toproceed with performance assessment, but it did

15

so through a series of research grants and onlyimplemented the assessments once they had beenthoroughly researched. What the results are usedfor also seems to make a difference. Most of thestates that report a lack of major difficulties inimplementing non-traditional assessments tend touse their assessments as end-of-course exams (forexample, Alabama's Math End-of-Course Test andCalifornia's Golden State Exams), for early-childhood screening (for example, Georgia'sKindergarten Assessment Program), forcareer/employability skills assessment (forexample, California's Career-TechnicalAssessment Program), as instructional planningtools (for example, Connecticut's AcademicPerformance Test), or when the alternativeassessment is a writing sample (for example,Idaho's Writing Assessment, Rhode Island'sWriting Assessment, and Vermont's Uniform Testin Writing). All of these are fairly low-stakespurposes, meaning that consequences of poorperformance are not severe for students, schoolsand/or teachers. State assessments seem to comeunder attack most often when the use of the testresults is high-stakes student graduation, schoolaccreditation, school takeover, etc. Of course,these assessments receive the most press attentionand public appraisal. Most programs have flaws,but when severe consequences are dependentupon the results, any flaw becomes morepronounced.

However, it is clear that strictly traditionalprograms are becoming increasingly uncommon.States are embracing new forms of assessmentand looking for ways to make them work. It's anexciting time in large-scale assessment, and aslong as the public is brought along, the technicalquality issues continue to be resolved, and the costand time management issues are addressed, ablended assessment program will continue to bethe preferred model of state assessment Goingback to the days of "one type fits all" assessment ishighly unlikely.

22

Chapter FourSpecial Topics: Assessment of Students with Disabilities and English Language Learners

and Title I Assessment and Evaluation Plans

Part I: Assessment of Students with Disabilities and English Language Learners

When the 103rd Congress overhauled theElementary and Secondary Education Act, thenew Title I legislation (the Improving America'sSchools Act) called upon states to hold allstudents to the same high expectations and toensure they have equal educationalopportunities (Phillips, 1995). The definition ofthose high expectations and the design of theassessment system used to determine whetheror not students have achieved those highexpectations are left to individual states andlocal school districts. This has spurred agrowing debate over which students should betested and how that testing should beconducted. A major concern surrounds theinclusion of students with disabilities andLimited English Proficient students in statewideassessment programs.

Three questions are of paramount importancein understanding the current practice ofassessing students. How many students withdisabilities and English language learnerscurrently participate in statewide assessmentprograms, what kinds of special testingconditions or accommodations are allowed toenable these students to participate, and aretheir scores included in public reports? Thesequestions were included in the Fall 1995 editionof the Association of State AssessmentPrograms (ASAP) survey. Additionalinformation about the assessment of studentswith disabilities is provided by the NationalCenter on Educational Outcomes (NCEO), agroup committed to assisting states inimplementing activities to improve outcomesfor these students, and to documenting states'efforts in doing so (Ysseldyke, 1996). Theauthor also relied heavily upon an articlewritten for NCREL by Susan Phillips, anattorney and measurement professor atMichigan State University, entitled All Students,

17

Same Test, Same Standards: What the New Title ILegislation Will Mean for the EducationalAssessment of Special Education Students.

Overall Policy for Ensuring the Participationof Students with Disabilities and EnglishLanguage Learners in the State AssessmentProgram

States were asked to describe their inclusionpolicy for students with disabilities, and mostreported that the decision was left to thestudent's Individualized Education Plan (IEP)committee as to "whether or not the studentcould meaningfully participate." Some statesprovide guidelines specifying that if the studentis in the regular education program for at leasthalf of his or her instruction in that subject, thenthe student should be tested. Still, the IEPcommittee makes the final determination. TheIEP committee also determines whether or notthe student requires testing accommodations inorder to participate, with an increasing numberof states stipulating that the testingaccommodations must be consistent with thosemodifications which are already used in thestudent's instructional program. The decisionto include or exclude a student., and the testingaccommodations to be allowed, must bedocumented in the student's IEP.

Some states have interesting twists on thisgeneral policy. For example, until this year,Kansas allowed students with disabilities to betested at either chronological grade level orinstructional grade level. This year, theinstructional grade level will no longer be anoption. North Carolina is also interestingbecause it attempts to encourage schools toinclude as many students as possible in theirstate assessment program by allowing onlythose schools without excessive exemptions to

23

be eligible to receive bonus funding fordemonstrating exemplary growth. One concernwith which most states have struggled is whatto do about testing accommodations that affectwhat is assessed. In Virginia, results of testingwith accommodations that result in changes inthe construct being assessed (e.g., listening, inplace of reading) are flagged. Vermont allowsstudents with disabilities to be excluded fromnorm-referenced assessment, but not from theirportfolio program.

Decisions to include or exclude Englishlanguage learners from a state assessment alsotend to be made at the local level, although stateguidelines tend to be fairly specific. In moststates, the decision to include an Englishlanguage learner is based upon the student'sEnglish language proficiency as determined bythe amount of time in an English as SecondLanguage (ESL) program and/or their score ona test of English proficiency. North Dakota'spolicy states emphatically that these studentsare not to be excluded. The state divides itsassessment results into subgroups to fosteraccountability for "all" students' learning.Tennessee has a very interesting policy forEnglish language learners; first-year Englishlanguage learners must take, at a minimum, themath computation sections of the stateassessment. Second-year English languagelearners must take the reading and languagesections in addition to math computation.Third-year English language learners mustcomplete the full battery. In Maryland, Englishlanguage learners may be exempted from stateassessment only once. In Vermont, Englishlanguage learners can be exempted fromstandardized, norm-referenced testing but notfrom the state portfolio.

For all states with a high school graduation test,the inclusion or exclusion of students withdisabilities or English language learners inassessment presents two added concerns,which are fairness to students and accuracy ofcertification. In other words, do students whoare excluded from the graduation test receive adiploma, and do students who take the high

school graduation test under non-standardizedconditions receive a special designation on theirdiploma that documents this? The first issue isobvious - should a student's disability impactthe student's ability to demonstratecompetence? Most would answer a resounding"no." However, the second question is harderto deal with. If the accommodation that isprovided to a student affects that student'sscore in a way that does not give the student anadvantage when compared to other students,the accuracy of the certification of competencecomes into question. Although we didn'taddress these issues specifically in the survey, anumber of states addressed these concerns intheir response to the general questions aboutinclusion and accommodation policies. Forexample, in Florida, only students who take thegraduation test may receive a regular diploma,and other students who do not take the statetest are encouraged to pursue special highschool diploma requirements.

Until Title I legislation called for the assessmentof all students, most states chose to leave thedecision about inclusion and accommodation tothe local district. The increased attention beingpaid to the topic is evident in the increasednumber of sessions on the subject at educationalconferences such as the American EducationalResearch Association (AERA) conference inChicago in March, and the Council of ChiefState School Officers annual assessmentconference in Colorado Springs, Colorado inJune. There is also a new Special Interest Groupon the Assessment of Special Populationsthrough AERA.

Participation of Students with Disabilities andEnglish Language Learners in StatewideAssessment

Forty-one states have written guidelines aboutthe participation of students with disabilities intheir statewide assessment programs, but of the133 different assessments employed by states,state special education directors participationcan estimate participation rates for only 49assessments (Ysseldyke, 1996). When

18

24

participation rates for students with disabilitiesare offered (by only 18 states), they range from3.5 to 15 percent of the total tested elementarypopulation and 0.4 to 12 percent of the totaltested high school population. The accuracy ofthese participation rates is questioned by bothstate testing directors and special educationdirectors because the data is not collectedsystematically in many places. Even with ourimproved questions, the decision aboutwhether or not to include a student withdisabilities in the state assessment programtends to be a local IEP committee decision. Thestate may not even know how many studentswith disabilities are exempted from testing.

This data is even less well understood forEnglish language learners where states estimatethat from 0.2 to 6 percent of the elementarystudents and from 0.2 to 5 percent of the highschool students who are assessed are Englishlanguage learners.

Better and more precise information about theassessment of students with disabilities andEnglish language learners will need to becollected at the state level in order to have anaccurate estimate of the participation rates ofthese students.

Survey Results Concerning the Inclusion andExclusion of Students with Disabilities andEnglish Language Learners

Chart 4-1 shows that 34 states allow somestudents with disabilities to be excluded fromall state assessments, 10 states allow for theirexclusion for some assessments, and only onestate disallows exclusions (Kentucky). WithEnglish language learners, 27 states allowexemptions from all state assessments, 11 statesallow it for some assessments and six states donot allow exemptions. See Chart 4-2.

BEST COPY AVAILABLE19

Chart 4-1Students with Disabilities:

Exemptions and Accommodations Policy

Allowed for AllAssessments

Allowed for SomeAssessments

Allowed for NoAssessments


*Exemptions OAccommodationd

Chart 4-2English Language Learners:

Exemptions and Accommodations Policy

Allowed for AllAssessments

Allowed for SomeAssessments

Allowed for NoAssessments


El Exemptions OAccommodations

Testing Accommodations Allowed

When students with disabilities or Englishlanguage learners are included in statewideassessment, the extent to which testingaccommodations are allowed also varies. FromCharts 4-1 and 4-2 we see that only one stateincludes students with disabilities in stateassessment programs withoutaccommodations, and 38 include them butallow accommodations. Five more states allowaccommodations for some assessments. Sixstates include English language learnerswithout accommodations, 11 states allowaccommodations with some assessments and 20include them in all assessments withaccommodations. For most of these decisions,if the assessment is deemed inappropriate, thatis, if the student is not expected to master the

25

content of the assessment as part of thestudent's instructional plan, a decision may bemade to exclude him or her from theassessment. If the assessment is deemedappropriate as is, the student is includedwithout accommodation. If the assessment isdeemed appropriate but only with specialaccommodations, the student is allowed thoseaccommodations. The decision is never asclear-cut as this sounds. A great deal of localflexibility is allowed in most states, and localdistricts interpret the broad state policies invaried ways.

Types of Testing Accommodations Allowed

This year, we divided allowable testingaccommodations into five categories forstudents with disabilities and English languagelearners: presentation format, setting, timing orscheduling, response format and other. Chart4-3 shows the kinds of accommodationsallowed for students with disabilities and forEnglish language learners.

Chart 4-3Testing Accommodations Allowed

Presentation Format

Setting

Timing or Scheduling

Response Format

Other


IEStudents with Disablities

=English Language Learners

Forty-two states allow students with disabilitiesto have accommodations in the way the test ispresented to them. Typical accommodationsinclude those offered to blind students orstudents with low vision such as large print orBraille editions of the test, and the reading,interpretation or repetition of test directions.For Limited English Proficient students,common accommodations also include the

20

reading, interpretation and repetition ofdirections but in addition include the translationof the directions and/or the test items into thestudent's native language.

The setting in which students with disabilitiesare assessed can be modified in 39 states. Asmight be expected, the kinds ofaccommodations that are allowed include quietsettings that are free from distractions, smallgroup settings, and/or administration bysomeone with whom the student is familiar.English language learners are allowed the samekinds of accommodations.

Changes in the timing or scheduling ofassessments is allowed for special educationstudents in 38 states. Extended testing time,more frequent breaks, and the spreading out oftesting sessions across several days are typicalmodifications in timing and scheduling. Similaraccommodations are allowed for Englishlanguage learners, although not as often.

Students with disabilities are allowed tochange the way in which they respond to theassessments in 36 states. Students can use acomputer or word processor or have a scriberecord the students' responses. With Englishlanguage learners, students may also beallowed to respond in their native language.

In the other accommodations categories,some states allow students with disabilitiesand English language learners to use wordlists or dictionaries, and/or allow out-of-leveltesting for students with disabilities.

Public Reporting of Assessment Results

Even when special education studentsparticipate in the statewide assessmentprogram, their scores may not be included inthe state, district and school averages. Manystates offer schools this option, partly becausethey are interested in having as many specialeducation students tested as possible, andpartly because the special circumstances underwhich some special education students take the

26

test make the results less comparable to those ofother students.

Chart 4-4Public Reporting Policy for Students with

Disabilities and English Language Learners

Included in one ormore regular reports

Excluded from all reports

Included in one ormore separate reports


629Students with Disablities

0 English Language Learners

More states include the scores of Englishlanguage learners in their regular public reportsthan those of students with disabilities (SeeChart 4-4). A similar number of states (12 forstudents with disabilities and 10 for Englishlanguage learners) exclude the scores ofstudents with disabilities and English languagelearners from all public reports. Less than halfas many states (four) provide separate reportsfor English language learners than do states forstudents with disabilities (nine states).

What Next?

While the field of measurement has contributeda set of rules concerning reliability and validityof results that help govern the inclusion andaccommodation of students with disabilitiesand English language learners, little actualresearch exists that demonstrates the impact ofaccommodations on test validity. Severalstudies are underway, a number of themsponsored by the National Center forEducational Statistics (Phillips, January 19,1996), to address this question empirically. Inaddition, special education and assessmentrepresentatives from thirty states met inJanuary 1996 at a CCSSO Special EducationState Collaborative on Assessment and StudentStandards (SCASS) to discuss these and otherrelated questions concerning the assessment ofspecial education students. Other studies areneeded to assess the impact of inclusion orexclusion of special education and Englishlanguage learners on the educationalopportunities these students receive as a resultof that decision. The studies now underwayshould provide additional guidance to thosewho are concerned for the right of thesestudents to be assessed and to be provided theopportunity to reach the same high standardsas their non-disabled or English language peers.

21 27

Part II: State Title I Assessment and Evaluation Plans

A separate section of the Fall 1996Association of State Assessment Programssurvey was dedicated to states' assessmentand evaluation plans for Title I. The majorissues states are dealing with are the revisionof their academic content standards, thesetting of performance standards to reflect"rigorous standards for all students," and thedefinition of adequate yearly progress forprogram evaluation purposes. They are alsostruggling with how best to include allstudents in the assessment program, an issueaddressed in the previous section of thischapter.

Content and Performance Standards

Chart 4-5 clearly indicates that most states areactively engaged in the revision of state goals,content and performance standards, andcurriculum and assessment frameworks. Muchof this activity was underway even before theTitle I legislation was changed in 1995, but theactivity has accelerated since that time.

Chart 4-5State Goals, Standards and Frameworks

Academic Content Standards

Performance Standards

Curriculum Frameworks

Assessment Frameworks

State Goals

None of the Above


The requirement to establish two performancestandards for students and schools ("proficient"and "advanced") was reported as a concern bystates in last year's annual report. Chart 4-6shows that states are farther ahead in this forthe development of performance standards for

22

students (they've done so with 35 stateassessment components) than for schools (20state assessment components). There are 29state assessment components for which studentperformance standards have not been set and33 components for which school performancestandards have not been set.

Chart 4-6Peformance Standards Setting

For Students

Have been seton a component

Have NOT been seton a component

For Schools

Have been seton a component

Have NOT been seton a component

0 10 20 30 40 50

Number of States

Adequate Yearly Progress

Once school performance standards have beenset, the state needs to define adequate yearlyprogress for the school. The state has todetermine whether or not the school hasadvanced the learning of Title I students (or allstudents in a school-wide program) sufficientlyto satisfy the "adequate yearly progress"designation. According to Chart 4-7, 34 stateshave defined this, while 13 states have not.Whether or not these definitions are finaldefinitions is less certain.

28

Chart 4-7Definition of Adequate Yearly Progress

for Title I Purposes Status

Not Defined

Has Defined

If Defined. Final?

Yes

No

Undecided


Status of State Title I Assessment Plans

States are currently required to havetransitional Title I Assessment andEvaluation Plans in effect, but they will needto have final plans in place by the year 2000.Chart 4-8 displays where states are indesigning their Title I assessment plans.

Chart 4-8Status of State Transitional and Final

Title I Assessment Plans

One or more currentcomponent used in plan

Not using any currentcomponent in plan

Undecided about usingany current component


*Transitional Plan DFinal Plan I

Approximately 60% of the states are usingone or more of their current components fortheir transitional plans, while fewer statesplan to use one or more of their currentcomponents in their final plans.Interestingly, norm-referenced tests, criterion-referenced tests, performance assessments,and writing assessments are all included in

both transitional and final plans. Ten statesreport that they will not use any of theirexisting assessment components in their finalTitle I assessment plan. This indicates thatapproximately 20% of the states need todesign or are in the process of designing atotally new assessment system.

Summary

There is so much assessment activityunderway in the states that final Title I plansare still up in the air. New content standardsare driving changes in assessment contentand format, and until these changes are fullyimplemented, decisions about final Title Iplans will not be made. Compared with lastyear's survey results, states are clearlymaking progress. However, much remains tobe done, especially in the areas of settingschool standards and defining adequateyearly progress.

BEST COPY AVAILABLE

23 2 9

Chapter FiveStatewide Assessment History and Trends

Introduction

This is the fifth year in which the informationabout statewide student assessmentprograms has been collected systematicallyand made available by CCSSO and NCREL.With data being collected for this number ofyears, it is possible to begin to see trends inthe information. Although this sectionreports trends over a five-year period, trendsstill must be interpreted cautiously, sincechanges in student assessment programs takeseveral years to conceptualize andimplement. It is unlikely that substantialchange will take place from one year toanother. However, the information reportedhere is similar to information collected lessformally in the past, so that it is possible tocombine current detailed information withpast information to perceive longer-termtrends.

The purpose of the following sections is tocomment on some of the changes that haveoccurred since the early 1970s, with particularemphasis on trends since 1991. In addition,several issues that may have implications forlarge-scale assessment in the future arementioned.

Criterion-Referenced Assessment andMinimum Competency Tests

When the Association of State AssessmentPrograms (ASAP) was formed in 1977 torepresent the assessment programs at thestate and national levels, two stronginnovations had occurred and were beingspread throughout the states. First, statessuch as Michigan had adopted a new form ofmeasurement called "criterion-referencedtests" in the early 1970s. Scores werereported as pass-fail for individual objectivesas well as proportion of the objectives passed,rather than comparing student (or school or

25

district) scores to national norms. Second,states began to use tests to determinewhether students had learned enough toreceive a high school diploma. This use ofminimum competency testing for high schoolgraduation was exemplified by the landmarkprogram in Florida.

ASAP was formed by the states to assist themin developing quality assessment programswith the minimum of wasted effort andcontroversy. Early ASAP meetings werefilled with discussions about the proceduresfor developing criterion-referenced tests, aswell as surviving the inevitable legalchallenges to the minimum competency tests,since the landmark legal case Debra P. v.Turlington was occurring then.

The predominant form of large-scaleassessment at that time was norm-referencedtests. Interest in criterion-referenced testswas pushed along not only by the states thatwere using them, but also by the NationalAssessment of Educational Progress (NAEP)in its early years, since several states (such asCalifornia, Connecticut, Minnesota andWyoming) gave the early NAEP assessmentsin "piggyback" style to obtain state andnational data on their students. Not only didthis practice introduce these states tocriterion-referenced testing, it also served asan introduction to the concept of the stateNAEP assessment program.

Advent of Writing Assessment

In the 1970s, assessment was limited usuallyto mathematics and reading. NAEP,however, added assessment in a number ofadditional subject areas, including science,social studies, citizenship, music, visual arts,career and occupational development,literature, and writing. Assessmentsincluded both conventional approaches such

30

as multiple-choice items, and lessconventional approaches such as constructed-response and individually administered,hands-on performance measures. Many ofthese innovative approaches to assessment(and even the subject areas) were droppeddue to budget cuts in the 1970s, but thecontinued use of performance measures inwriting remained.

The NAEP assessments of writing in the early1970s encouraged the belief that having allstudents at one or more grade levels actuallywrite essays would be feasible. Althoughmore expensive than the much moreprevalent multiple-choice tests of "writing,"essay tests were thought to be more contentvalid, and it was believed that they wouldlead to better teaching of writing. Substantialdebate about the cost versus the benefit ofthis type of assessment occurred in the 1970s,but the belief in the use of performanceassessment was so strong that it remains theprevalent form of assessment used in writingby states.

Expansion to Other Subject Areas

During the 1980s, additional states adoptedlarge-scale assessment programs as a tool forschool reform and improvement. Each yearat the ASAP meetings, one or two states newto large-scale assessment efforts wouldattend. In addition, states were beginning toadd other subject areas to their assessments.They began to develop assessments in areassuch as science, social studies (or one or moreof its components, such as history orgeography), health education, physicaleducation, the arts, and vocational education.Interest also grew in the sharing ofassessment items or tasks among the states,since so many new states were nowinterested in large-scale assessment.Attempts were made to create item banksamong the states. These generally provedunsuccessful, since each state clung to its own

26

set of student expectations, making sharingof corresponding items challenging at best.


The latter part of the 1980s also broughtattention to the form of assessment.Multiple-choice tests were (and still are) themajor form of assessment used in most states,except for states that used a writing sample.In the last few years, a couple of trends havestarted. First, a small group of states(Maryland, Kentucky, California, andArizona being first, now joined by Maine) hasbegun to use performance assessment(constructed-response formats) as theprimary form of assessment. Other states areconsidering developing such programs,including Massachusetts and Delaware.These states demonstrated that it is feasible toadminister alternative forms of assessment ina relatively cost-effective manner. However,concerns about test content caused theinnovative assessment program in Arizonaand California to be shelved and causedKentucky to add a norm-referenced test to itsassessment program.

Second, a number of states are working on orpiloting alternative forms of assessment. Thisinnovative work includes performanceassessments that are given to individuals orsmall groups of students; curriculum-imbedded tasks in which assessment isintricately interwoven within teaching andassessment information is collected overseveral weeks or months; the use of portfoliosto collect examples of student work for laterscoring; and other innovative forms ofassessment. As the SSAP survey indicates,few states have actually implemented theseinnovative alternative forms of assessment.Given the number of states reporting suchwork, these numbers might increase in thefuture. It is likely that, given the costs ofalternative assessment in money and time,most states will move toward the concept ofan assessment system using different forms

31

of assessment at different levels. Forexample, large-scale, standardizedassessments with some alternativeapproaches might be used for state-levelreporting, while more extensive programs ofperformance and/or portfolio assessmentmight be used to meet school or classroomassessment needs. Oregon is an illustrationof such a program. Several states report thatsuch innovative performance assessments arebeing developed for use by local educators.

States thinking about innovative approachesto assessment are challenged to consider thecosts (both financial and instructional time)involved in using such measurementstrategies, as well as the technical concernsabout these new approaches to assessment.Although they have a strong advantage ofillustrating better approaches to teaching andlearning, alternative assessments may be lessreliable for reporting individual student orschool results, and certainly are moreexpensive. Therefore, in recent years, severalstates have considered the use of a "mixed"assessment model in which students areassessed with a combination of multiple-choice and open-ended exercises. Thisapproach has the advantage of allowingstates to assess more content but at lower costthan an entirely open-ended assessment.Kentucky is using this approach, andMassachusetts is considering it.

Another approach to broader contentcoverage is the use of every-pupil matrixsampling designs. This approach is usefulwhere school and district information aremore important than individual studentresults. Kentucky and Maryland have usedthis approach for several years.

Professional Development on Assessment

Attention to the forms of assessment used atboth the state and local levels has encouragedanother trend at the state level. As stateeducators have debated the appropriate

27

forms of state assessments, they haveincreased attention on the training ofclassroom teachers to collect and useinformation collected in their classrooms.This trend is actually the convergence ofseveral trends, including changes in studentstandards in order to emphasize thinking andproblem-solving skills (while de-emphasizingmemorization of content knowledge), plussupport for alternative approaches toassessment, such as projects, exhibitions,demonstrations and the use of portfolios.The result is that many local districts andsome state agencies are now providingclassroom teachers with assessment learningexperiences that teachers can apply in theirclassrooms. Washington State illustrates thisactivity quite well. This attention toprofessional development on assessment forclassroom teachers is particularlyappropriate, given that few if any teachersreceive much in the way of pre-servicetraining on assessment. The SSAP surveyreports considerable attention to the use oftest results by states.

Norm-Referenced Tests

When ASAP began meeting in 1977, the mostcommonly used assessments werecommercially available, off-the-shelf, norm-referenced tests. Despite the attention toforms of measurement such as criterion-referenced assessments, which are morewidespread today than 20 years ago, norm-referenced tests are still a predominant formof large-scale assessment. In recent years,there has been a slight decrease in the use ofnorm-referenced tests at the state level. In1993, 31 states used norm-referenced tests, 30reported using them in 1994, 31 statesreported using them in 1995, and 29 statesreported using norm-referenced tests in 1996.

There had been an expectation that thisnumber would fall even further, given the de-emphasis on norm-referenced assessments inthe Improving America's Schools Act (IASA),

32

the reauthorization of the Elementary andSecondary Education Act. States are nolonger required to use such assessments forthe evaluation of Title I compensatoryeducation programs, nor the monitoring ofindividual Title I student improvement. Thiswas a major change in the legislation, whichadvocacy groups and others fought for andwon. Instead, states are required to developand operate "comprehensive assessmentsystems" capable of reporting whetherindividual students and school programs aremaking "adequate yearly progress."

Two events conspired to confound thisprediction. First, the November 1994 electionbrought to power chief state school officers,state board of education members, legislatorsand governors with strongly-held ideas aboutstudent standards and assessment and withdeeply held concerns about the new forms ofassessment and their standards. In addition,policymakers and the public wanted nationalcomparisons. Given problems in some of theassessment efforts first implemented (i.e., inArizona, California and Georgia), policy-makers pushed to set aside innovativeapproaches to assessment and to return tocommercially-available, norm-referencedtests. States such as Alabama re-implemented norm-referenced tests inaddition to their criterion-referencedprograms.

Second, the changes implemented in theIASA legislation have proven to be less far-reaching than originally thought. Due topolitical pressures, states will be required tochange their statewide assessmentssubstantially less than originally thought.States, for example, have five to six years todevelop a final comprehensive assessmentsystem (and only in mathematics andreading, not in all of the national goal areas,unless they do so for other students). In theinterim, transitional assessments (norm-referenced, criterion-referenced, orperformance assessments) can be used by

28

states, so long as they "measure challengingstate content standards," which is poorlydefined in the federal legislation.

For these reasons, and because manypolicymakers desire to have comparativedata using test instruments developedoutside of states, norm-referenced tests likelywill continue to be a major type ofassessment used in states. To satisfy thisdesire for normative information, but usingmeasures of higher-level standards, somestates (such as Kentucky and North Carolina)have administered the National Assessmentof Educational Progress (NAEP) assessmentsto samples of students taking their statewideassessments in order to provide NAEP-likescores to buildings and districts (as well asthe state). This recent innovation inproviding normative information has thepromise of allowing states to pursue newforms of assessment while still providingexternal referents for scores on the statewideassessments. It is even possible that someform of individual student NAEP test mightbe made available as well. It will beinteresting to monitor the success of theseefforts and to determine if this becomes atrend for the future.

In early 1997, President Clinton proposed avoluntary national test of fourth gradereading and eighth grade mathematics. Hisproposal was similar to one proposed in 1996by Governor Engler of Michigan, then co-chair of the National Education Goals Panel.The President's proposal calls for thedevelopment (at federal expense) of a mixedassessment based on (but different than) theNAEP frameworks in these subjects, to yieldindividual student scores, and to be given atfederal expense during its inaugural year of1999. States would be free to sign up to giveit, and several have proposed methods oflinking the assessment to their currentassessments, not unlike how Kentucky andNorth Carolina linked the NAEP assessmentsto their state assessments in the past.

33

The national test has been warmly receivedby some who view it as a means of gaugingstate performance on national contentstandards, avoiding some of the pitfalls seenby critics in the use of commercial norm-referenced tests. However, several stateshave expressed concerns about the viabilityof both the NAEP assessments and their ownstatewide assessments. The impact of thevoluntary national tests has just begun to befelt, however, so it is too soon to tell what itslonger term impact will be on states or thetesting industry.

National Efforts at Joint Development

Another trend is worth noting. Until 1990,most assessment development was carriedout by individual states working alone orwith the assistance of a contractor. Sincethen, two innovations in collaboration amongthe states have taken place. The first is theNew Standards, co-directed by the Universityof Pittsburgh Learning Research andDevelopment Center and the National Centerfor Education and the Economy, which hasbeen working with a number of states andlocal districts to design and develop aninnovative assessment system that willencourage thoughtful student learning inareas such as mathematics, language arts andscience.

The second is the Council of Chief StateSchool Officers' State Collaborative onAssessment and Student Standards (SCASS),which currently has 11 projects in whichstates work together to design or developinnovative student assessments. Both ofthese activities mark a first for collaborationamong the states. The states are activelyworking together to develop assessmentsfrom which states share and use theproducts, rather than simply exchanginginformation about innovative assessmentapproaches, as has been the case in the past.

Accommodating Students with Disabilities

One issue that states appear now to be facingis how to provide accommodations forstudents with disabilities on many of thestatewide assessment components. Muchwork appears to be taking place to encouragelocal school districts to use variousaccommodations to increase the participationof these students, for several reasons. First,advocacy groups for these students havemade participation in large-scale assessmentprograms a major issue. Second, federalagencies (the Offices of Special EducationPrograms and of Educational Research andImprovement of the U.S. Department ofEducation) have provided substantialfunding to states to investigate methods andprocedures for accommodating students withdisabilities. States have used funds todevelop new assessments, to develop newaccommodations policies, to research theimpact of these accommodations policies, andto develop materials to explainaccommodations. Third, nationalorganizations such as the National Center forEducational Outcomes of the University ofMinnesota have been tireless in their effortsto assist states in providing accommodationsto all students. They have partnered withorganizations such as CCSSO to workdirectly with state assessment and specialeducation staffs in design work onaccommodations.

All of this work has resulted in a wide varietyof accommodations being provided by stateswith many of the assessment componentsrun by states. The categories ofaccommodations contained in the surveyinclude accommodations regardingpresentation format, test setting, responsesetting, timing/scheduling and others.Interestingly, the accommodations permittedin one state may not be permitted in otherstates.

29 34

While many states offer a broad number ofaccommodations, few provide alternativeassessments for students unable to useavailable accommodations to participate inthe large-scale assessment program. Only sixstates reported providing alternativeassessments (such as alternative portfolioassessments) that collect and reportinformation on students. Of course, suchalternative assessments raise various policyand technical issues.

Assessing English-Language LearningStudents

As much work as there appears to be inproviding accommodations for students withdisabilities, much less work appears to beoccurring in assessing students learningEnglish (limited-English proficiency). Morestates report permitting exclusions ofstudents from large-scale assessmentrequirements, and fewer accommodations areprovided for these students. In addition,fewer alternative assessments are provided.Clearly, this is an area in need of both policyand technical development among the states.A CCSSO project is at work in developingappropriate assessments andaccommodations for these students.

Future Issues and Their Impact on StateAssessment

Overall, an examination of the changes inlarge-scale assessment programs during thepast 20 years shows a substantial change inthe number of states with such programs, thesubject areas assessed, and the types ofassessment measures used - as well as thetypes of assessment measures beingdeveloped (and the manner in which thisdevelopment is proceeding). These changeshave only increased in the past few yearswith the considerable public attention paid tothe quality of schools.

30

Not surprisingly, these changes have led anumber of states to re-examine assessmentprogram designs that were adopted in yearspast. A number of states are examiningwhether their current assessment designs arestill adequate and are looking at how suchprograms as NAEP, the New Standards,SCASS, IASA Title I and the national test fitwithin their overall assessment design.Given the number of states that areconducting such reviews, further changes inthe nation's large-scale assessment programsare likely. Of course, it may take severalyears for these changes to be implemented.

Several trends appear at the state and locallevels that may have a long-term impact onthe shape of large-scale assessment programsat the state level. Certainly, the currentemphasis on performance or alternativeassessments is not going to disappear.Although there have been some successes(such as in Maryland and Kentucky), the set-backs in California, Arizona, Indiana andelsewhere indicate that widespreadacceptance of performance assessment is notautomatic. Technical issues need to beaddressed in a sound manner, and policy-makers and the public need to understandthe reasons for such measures, the studentstandards that they measure, and the reasonswhy both innovative standards andassessments are needed. States and othersinterested in innovative forms of assessmentwill need to make sure important parties are"on board" before engaging in this newdevelopment work. More than rhetoric isneeded.

Certainly, there will be some impact of thedrive now under way in some states to "de-regulate" public education and return controlof it to local school districts. While this driveis taking several forms, it would not beunexpected for these pressures to affect theextent and types of student assessment in thefuture. In some states, this may mean lessattention to statewide student expectations

35

and measures, while in other places, it maymean just the opposite.

The pressure to provide appropriateassessment training and experiences toclassroom teachers is also not likely to abate.The collaborative work across states is likelyto spread innovative approaches toassessment more quickly than it has in thepast. In addition, the outside politicalpressures to use assessment as a tool forreform of schools is not likely to lessen.Changes brought about by federal legislationsuch as Goals 2000 and IASA will occur aswell, but perhaps at a slower pace than oncethought.

Finally, the reauthorization of the NAEPprogram brought several changes that alsomay affect states. In recent years, NAEP hasoffered the trial state-NAEP programs, butunfortunately, recent appropriations for theprogram plans have not permitted a full-scalestate-NAEP program to be offered. Duringthe past year, the National AssessmentGoverning Board, the policy-making boardfor NAEP, has suggested a number ofchanges to NAEP. It is uncertain at this pointhow many of these changes will beimplemented for NAEP, what the shape ofthe program will be in the future, nor howthe NAEP of the future will affect states. Ifthe program is funded at a higher level, itmight affect the number of states thatadminister norm-referenced tests to studentsat one or more grade levels, since the NAEPdata could provide the types of nationalcomparisons that some states desire that aremore current, less expensive and moretechnically sound (particularly if NAEP datacan be returned more quickly and in moreuseful formats). In addition, it is uncertainhow the drive to develop a voluntarynational test will affect NAEP, the use ofnorm-referenced tests, or states' large-scalestudent assessment programs.Many swirling, cross-cutting trends at thestate level are affecting large-scale assessment

31

programs, and it is likely that these trendswill affect the nature of statewideassessments in the future. With the annualState Student Assessment Program survey, itshould be easier to track the course ofchanges in large-scale assessment programsat the state level. Future editions of thisreport will continue to indicate just how suchchanges are occurring.

36

Resources

Bond, L.A. & King, D. State High School Graduation Testing: Status and Recommendations. Oak Brook, IL:North Central Regional Educational Laboratory, 1995.

Mehrens, W. Issues and Recommendations Regarding Implementation of High School Graduation Tests. OakBrook, IL: North Central Regional Educational Laboratory, 1992.

Mehrens, W. Issues to Consider in Moving Beyond a Minimal Competency High School Graduation Test.Greensboro, NC: Southeastern Regional Vision for Education, 1995.

National Governors Association. 1996 National Education Summit Policy Statement. Washington, DC:1996.

Phillips, G. The National Center on Education Statistics: Development Work and Research on Adaptations andAlternative Assessments for Students with Disabilities. A presentation given at a Council of Chief StateSchool Officers SCASS Special Education Assessment Meeting in Tampa, Florida, January 19,1996.

Phillips, S. E. Legal Implications of High-Stakes Assessment: What States Should Know. Oak Brook, IL: NorthCentral Regional Educational Laboratory, 1993.

Phillips, S.E. All Students, Same Test, Same Standards: What the New Title I Legislation Will Mean for theEducational Assessment of Special Education Students. Oak Brook, Illinois: North Central RegionalEducational Laboratory, 1995.

Ysseldyke, J. Overview of the Issues of Participation and Inclusion of Students with Disabilities in Large-ScaleAssessment Programs. A presentation at a Council of Chief State School Officers SCASS SpecialEducation Assessment Meeting in Tampa, Florida, January 19, 1996.

33 37

Sum

mar

y T

able

: Sta

tew

ide

Ass

essm

ent P

rogr

ams,

Sch

ool Y

ear

1995

-96

#C O

KG

rade

sS

ubje

ctV

AN

CL

SS

opR

RW

erM

WW

PS

rT

est T

ype P

PIte

m T

ype

SE

EP

ST

estin

g P

rogr

am P

urpo

ses

IS

SH

SS

HA

tmP

t tH

ES

cPS

cS

TM

g1

23

45

67

89

10 1

112

AR

WM

cSc

tT

TA

r tC

RR

TW

jDpE

AP

DD

GA

Rkr

AK

1-

--

-A

--

-A

- -A

-AA

-A

--

--

A-

--

A-Y

Y-

-A

L6

--

-AA

AA

AA

AA

AA

AA

AA

AA

A-

AA

AA

-A

AA

- -

-YY

Y-

--

-Y-Y

- -

AR

2-

--

-AA

-A-

- A

A-

AA

AA

A-

--

AA

A-

-AA

A-

- -Y

YY

Y-

-A

Z1

--

--

A-

-A

--

A-

-AA

-A

--

- -A

--

--A

YY

YY

.-

CA

2V

VV

V-

- -V

VV

V-L

LLLL

LLLL

LLy

YC

O0

CT

2-

--

-A

-A

-A

-A

--

-AA

AA

- -

--

AA

A-

AA

AA

- -Y

YY

Y-

--

-YY

- -

DE

1-

--

A-A

- -A

-A-

--

-A

--

A-

--

-A

.-

-YY

YF

L3

--

--

A-

- -A

-AA

--A

AA

- -

- -A

AA

--A

-A

--

--Y

Y-

--

-Y

--

--

GA

5A

.-A

- A

- -A

- -A

AA

AA

AA

A-A

AA

AA

-AA

AA

--Y

YY

--

--Y

-Y-

-H

I3

--

-A-

-A

- A

- A

AA

AA

-A

--

--

AA

-V

-A

YY

Y-

--

-YY

Y-

-IA

0ID

3-

--A

AA

AA

AA

AA

-AA

AA

AA

--A

-A

A-A

AA

-A-

-YY

--

IL1

--

-AA

-AA

A-A

A-

- A

AA

AA

- -A

AA

--A

-A-

--

YIN

1-

--

A-

-A

--

-A

--

A-

-A

--

- -A

AA

--A

AA

--

-YY

YY

Y-Y

KS

1-

--

AA

A-

AA

-A

--

- A

AA

- -

--

- A

AA

-A

AA

- A

AY

YY

Y-Y

KY

2-

--

-A

--

-A

--

AA

--

AA

AA

A-

--

AA

--

-A

--

-YY

LA4

A-

-AA

AA

A-

- A

A-A

A-

AA

A -

AA

AA

- -A

AA

A-

-YY

Y-Y

--Y

YY

- -

MA

1-

--

-A

--

-A

-A

--

-A

- A

A-

--

-A-

A-A

AA

--

--Y

Y-

-M

D2

--

-A-

A-

AA

AA

AA

AA

AA

AA

--

- A

AA

-A

AA

A-

-YY

Y-

--

-Y-Y

YY

ME

1-

--

-A-

--A

--

A-

- A

AA

AA

- -

--

A-

--

-A

--

--Y

YY

MI

2-

--

-AA

-AA

- -A

--

A-

AA

-S

--A

AS

SA

AA

AS

--Y

YY

- -Y

-YY

-YM

N0

MO

2-

--S

-SS

-S-S

S-S

.-S

SS

S-

--S

S-

-S-S

--

--Y

YY

-YM

S3

--

--

AA

AA

AA

- A

-A

AA

A -

A-

-A

AA

A-

AA

A-

A-

YY

Y-

--

-Y-Y

YY

MT

1-

--

-A

--

-A-

-A-A

A-

AA

A-

-A

--

--A

-YY

NC

3-

--A

AA

AA

AA

A-

-AA

AA

AA

--S

AA

--A

AA

--

--Y

YY

Y-

-Y-

---

ND

1-

--A

- -

A-

A-

-A-A

A-A

-A-

-A-

--

-AY

YY

YN

E0

NH

1-

--

A-

-A

--

-A

- -A

-A

AA

A-

--A

A-

-AA

A-

--

-YY

NJ

2A

- -A

A-A

AA

--

--

-AA

- -A

AA

--

-YY

Y-Y

-YN

M4

-A

AA

AA

A-

A-

AA

AA

AA

AA

A-

AA

AA

AL

AA

AL

LL

yY

Y-

--

-Y

-Y

Y-

N V

3-

--

-A-

--

A-

-AA

AA

AA

---

-A-A

-A

. A-

- -Y

YY

--

--

Y-

Y-

-N

Y7

--

-AA

AA

-AA

AA

AA

AA

AA

AV

--

AA

A-

AA

AA

- -

-YY

YY

YY

Y-Y

- -

OH

4-

--

-A-

A-

AA

AA

A-

AA

AA

- -

--

AA

A-A

AA

--

--Y

Y-

--

-YY

Y-

-O

K2

--

-A-

AA

AA

A-A

AA

AA

AA

A-

-AA

A-

-A-

A-

- -Y

YY

Y-Y

OR

2-

--

A-

A-

-A

--

A-

-AA

A-

--

--

AA

A-A

-A

--

--Y

YY

--

PA

2A

S-A

S-A

--A

AS

- -

--

AS

SA

-A

- A

--

-YY

Y-

-R

I4

--

--

A-

--

A-

A-

--A

AA

- -

- -A

-A

A-A

-A

--

--Y

YY

--

SC

2-

--

AA

AA

AA

AA

A-

AA

AA

A-

--

AA

A-

-A

--

-A

-Y

YY

-Y

--

YY

YY

-S

D1

--

- -A

- -

-A

- -A

-AA

-A

AA

- -A

A-

--A

-YY

TN

4-

-A

AA

AA

AA

A -

AA

AA

AA

A-

-A

AA

A-

AA

--

-YY

Y-

- -Y

Y-Y

--

Tx

1-

--

AA

AA

AA

- A

AA

-A

AA

AA

- -

-AA

- -A

-A

--

-YY

YY

---Y

-Y-Y

UT

3-

VV

VV

AV

VA

VV

AV

AA

VV

VA

--

AV

VV

-A

VV

V-

-Y

YY

Yr

--

VA

2-

--

-A-

AA

AA

AA

AA

AA

AA

A-

-AA

A-

-AA

--

A-

YY

YY

Y-

-Y-Y

- -

VT

2-

--

-V

--

-V-

V-

--

-VV

- -

--

-V

-V

V-V

V-

- -Y

YY

Y-

-W

A1

--

--

A-

- -A

- -A

-AA

-A

AA

- -A

--

--A

YY

YV

.-

WI

2-

--

AA

--

-A

-A

--

AA

-A

AA

--

AA

A-

-AA

A-

--

--Y

Y-

-W

V3

-AA

AA

AA

AA

AA

A-A

AA

AA

A-

-A

AA

A-

AA

A-

--Y

YY

ON

1S

SS

SS

--

--S

--

--

-SS

-Y

YT

otal

107

23

424

32

22 2

3 18

39

16 3

0 35

16

38 2

8 34

43

30 2

66

329

33

36 2

34

4124

36

910

427

43

38 1

05

13

178

335

11

Not

e: C

olor

ado

(CO

), Io

wa

(IA

), M

imes

ota

(MN

), a

nd N

ebra

ska

(NE

) Id

not

adr

reris

ter

any

stat

ewid

e as

sess

men

ts In

199

5-96

.

BE

ST C

OPY

MA

U L

E39

Sum

mar

y T

able

Leg

end

Not

es:

Tot

als

are

com

pute

d by

add

ing

the

num

ber

of A

's, S

's, V

's, L

's, a

nd Y

's in

eac

h co

lum

n. In

any

cel

l, on

ly o

ne d

escr

iptio

n is

incl

uded

. If m

ultip

le c

ompo

nent

des

crip

tion

wer

e po

ssib

le fo

r th

e sa

me

cell,

then

this

cel

l was

cod

ed to

mat

ch th

e co

mpo

nent

test

ing

the

mos

t stu

dent

s. S

ee P

art I

II of

the

surv

ey o

f the

SS

AP

dat

abas

e fo

r co

mpo

nent

nam

es a

nd c

ompo

nent

-leve

l des

crip

tions

.

#CO

M=

Num

ber

of S

tate

Ass

essm

ent C

ompo

nent

s

Cel

l Con

tent

sS

ubje

cts

(indi

cate

s at

leas

t one

LA=

Lang

uage

Art

sco

mpo

nent

whe

re)

R=

Rea

ding

A=

All

stud

ents

are

test

edW

=W

ritin

gS

=S

tude

nts

are

sam

pled

M=

Mat

hV

=In

clus

ion

is v

olun

tary

Sc=

Sci

ence

for

stud

ents

, sch

ools

,or

sch

ool d

istr

icts

SS

=V

oc=

Soc

ial S

tudi

esV

ocat

iona

l Ed

L=A

sses

smen

ts a

relo

cally

det

erm

ined

Ap=

Apt

itude

s

Y=

Pur

pose

is c

heck

ed fo

rat

leas

t one

com

pone

nt

Tes

ting

Pro

gram

Pur

pose

s

Inst

ruct

iona

l Pur

pose

sS

tD=

Stu

dent

dia

gnos

is o

rpl

acem

ent

Imp=

Impr

ovem

ent o

fin

stru

ctio

nP

E=

Pro

gram

Eva

luat

ion

40

Stu

dent

Acc

ount

abili

tyS

tA=

Stu

dent

dia

gnoi

sor

pla

cem

ent

StP

=S

tude

nt p

rom

otio

nH

D=

Hon

ors

dipl

oma

ED

=E

ndor

sed

dipl

oma

HS

G=

Hig

h sc

hool

grad

uatio

n (e

xit

requ

irem

ent)

Tes

t Typ

eN

RT

= N

orm

-ref

eren

ced

test

ing

CR

T=

Crit

erio

n-re

fere

nced

test

ing

WA

= W

ritin

g as

sess

men

tP

er-

Per

form

ance

ass

essm

ent

Prt

=P

ortfo

lio A

sses

smen

t

Sch

ool A

ccou

ntab

ility

ScA

= S

tude

nt a

war

ds o

rre

cogn

ition

SP

R=

Pub

lic s

choo

lpe

rfor

man

ce r

epor

ting

HS

k= H

igh

scho

ol s

kills

guar

ante

eA

cr=

Acc

redi

tatio

n

Item

Typ

eM

C=

Mul

tiple

-cho

ice

item

sS

WR

= S

hort

writ

ten

resp

onse

EW

R=

Ext

ende

d w

ritte

n re

spon

seP

T=

Per

form

ance

Tas

ksE

SW

= E

xam

ples

of S

tude

nt W

ork

Prj=

Pro

ject

s

Order FormCH0e,

ANNUAL SURVEYof

State Student Assessment ProgramsCouncil of Chief State School Officers One Massachusetts Avenue, NW., Suite 703, Washington, DC 20001-1431 (202) 408-5505 voice (202) 789-0596 fox

To order the following items from the State Student Assessment Program database, please fill out the shipping information completely and enclose paymentto 'CCSSO.° The cost of each is based on the type of organization ordering the information. Indicate the quantities of each type of material desired.

Quantity Cost

Annual Survey of State Student Assessment ProgramsThe 285 page report containing the 1996-97 data tables.

Testing Company or Assessment Contractor ($199)O Non-Profit Organization ($49)

University Faculty or Student ($19)

Adobe Acrobat electronic version of the Survey ReportMacintosh Testing Company or Assessment Contractor ($199)

0 Windows 0 Non-Profit Organizations ($49)O University Faculty or Student ($19)

Paradox files of the Survey Report data(Windows only) Testing Company or Assessment Contractor ($250)

t Independent Researcher ($100)

Trends in State Student Assessment ProgramsA 36-page summary, containing an overview of the 1996-97 informationand changes since 1992-93, is available in print. ($10.95)

Adobe Acrobat electronic version of Trends PaperL1 Macintosh (Also $10.95)

O Windows

Shipping Information:

Shipping/Handling(Add $2 per item ordered; $10 for overnight shipment)

Total Payment Enclosed

Telephone Fax

Return this order form with your payment (payable to "CCSSO') to:

Council of Chief State School OfficersAnnual Survey of State Student Assessment Programs

One Massachusetts Avenue, N.W., Suite 700Washington, DC 20001-1431

42

(9/92)

U.S. DEPARTMENT OF EDUCATIONOffice of Educational Research and improvement (OERI)

Educational Resources Information Center (ERIC)

NOTICE

REPRODUCTION BASIS

ERIC

This document is covered by a signed "Reproduction Release(Blanket)" form (on file within the ERIC system), encompassing allor classes of documents from its source organization and, therefore,does not require a "Specific Document" Release form.

This document is Federally-funded, or carries its own permission toreproduce, or is otherwise in the public domain and, therefore, maybe reproduced by ERIC without a signed Reproduction Releaseform (either "Specific Document" or "Blanket").

document resume ed 412 278 author bond, linda; roeber ... · document resume. ed 412 278 tm 027...

Documents