Possible Evaluation Model:Reactions from the Field

David T. Conley, Ph.D.

Professor, University of Oregon

2

Summary of ResponsesSurvey Questions: Do you generally support or not support the concept of the following? Support Not Sure

Do Not Support

1) A statewide test in English, math, and science focusing on keystone standards? 14 1 0 2) Allowing districts to supplement the statewide test that is focused on keystone standards with items that reflect local priorities so that the results can be used for formative or diagnostic purposes locally? 11 3 1 3) Course-embedded assessments that contain items that are consistent statewide and the results of which contribute to the student grade in the class? 7 6 1 4) A limited number of course-embedded performance tasks that are scored locally using statewide scoring criteria?* 8 6 1 5) A mandatory no-stakes college-readiness exam in 11th grade?* 6 4 4 6) An extended application given in 12th grade and designed to assess one or more of the Essential Skills including the Career Related Standards? 10 4 1 7) The overall concept and purposes of the assessment system outlined above? 7 8 0

3

Some Key Observations Educators want certainty and consistency

regarding what is assessed Core standards State-level scoring criteria and guides

They expect the state to provide clarity in all areas where the state is establishing assessment requirements

4

Some Key Observations Educators are concerned about

comparisons among schools Not surprisingly, educators favor formative

assessment over accountability assessment

Educators are split on whether they want a great deal of discretion to develop some aspects of the assessment system locally

Many educators still believe the state should not be “tinkering in the classroom”

5

Some Key Observations Implementation challenges are fully

predictable, but complex and difficult nevertheless

Assessment, however well embedded in the classroom, will always be viewed as a departure from instruction by some, perhaps many, educators

6

Some Key Observations Issues related to special learner

populations are essentially the same under these options, but any new system provides an opportunity to address the needs of these students better and more appropriately

7

Some Key Observations Regardless of what is implemented,

the Board will face the realities of Oregon’s locally controlled schools Limited professional development

capacity locally No really effective networks of

educators Deprivation mentality

8

Some Key Observations The issue of what exactly constitutes

an “unfunded mandate” probably needs to be considered Is any part of a statewide assessment

system a local responsibility to fund and operate, or is the state responsible for all expenses related to assessment?

9

Some Key Observations Whatever choice the Board makes, it

will have to communicate its rationale and purposes clearly and unambiguously

At some point, decisive action will be required, and the field will never be fully ready for such decisiveness

10

Some Key Observations Undertaking any new direction in

assessment requires a long-term commitment from policy makers and policy implementers to work through the inevitable unforeseen problems that will arise

11

Some Key Observations The implicit assessment demands of

the diploma requirements are significant

The key decision is the degree to which all of the components will be assessed or whether some will be educational goals that are not necessarily measured

12

Some Key Observations Whatever option is pursued (or not

pursued), it is important to remember that the schools are suffering a CIM/CAM/TESA “hangover” and will view with extreme skepticism any assessment policy, regardless of its merits

13

Some Key Observations A key policy decision is the degree of

consistency the state wants in what is taught and learned in Oregon classrooms

The state has been unwilling to take a clear stand on this fundamental issue from the beginning of real statewide assessment in the early 1990s

14

Some Key Observations For educators, it’s all about the

details Even if consensus can be gained on

the general structure of the assessment system, much will hinge on myriad details of implementation

15

Some Key Observations The problem of conflicting

arguments: If the system is too simple, it’s criticized

for not being a valid measurement of what’s happening in the classroom

If it’s focused on validly measuring what’s happening in the classroom, it’s criticized for being too complex, time-consuming, expensive

16

Possible Simplification One possibility is to have some sort of key

assessment activity each year of high school that provides a range of info specific to diploma requirements College readiness test Term paper Research paper Senior project

State would set the conditions of each NCLB test would be standard for reading,

math

17

Summary While respondents are cautiously

supportive of the options, it’s all about the specifics and the details: Rationale for each requirement Uses of the data Implementation schedule Professional development resources Alignment with current district practice Time requirements Special needs populations

18

Recommendation The Board would need a very detailed

operational plan that addressed the following for any new assessment: Political support and sustainability Detailed operational requirements Adequate funding Professional development needs Involvement and acceptance by key

constituencies

19

Assessment Type by Purpose

Federal State Local Formative Accountability

State Test X X

X (federal, state)

Supplementary items on state test

X X

Course-embedded assessments

X X X (local) X (state)

College readiness measure

X X X

Extended application

X X

20

Essential Skills by Assessment Type

State Test

Supplementary items on state

test

Course-embedded assessments

College readiness measure

Extended application

Read and interpret a variety of texts

X X X

Write for a variety of purposes

X X

Speak and present publicly

X

Apply mathematics in a variety of settings

X X X

Demonstrate global literacy

X X

Use technology

X X

Think critically and analytically

X X X X

Demonstrate civic and community engagement

X

Career-Related Learning Skills Personal management

X X

Teamwork X Employment foundations

X

Career developme nt

X