national conference on student assessment june 21, 2013 ccsso 2013 presentation1

National Conference on Student AssessmentJune 21, 2013

June 21, 2013CCSSO 2013 Presentation 1

One state’s model for using published authors to generate quality reading texts for next-generation assessments◦ Michigan’s vision◦ Process and training◦ Authors’ perspectives

June 21, 2013CCSSO 2013 Presentation 2

Andrew Middlestead, Test Development Manager, Michigan Department of Education

Kara Courtney, Data Recognition Corporation

Michael P. Spradlin, Author Shirley Neitzel, Author Patty McDivitt, Data Recognition

Corporation

June 21, 2013CCSSO 2013 Presentation 3

—Andrew Middlestead Office of Standards and Assessment in the Michigan

Department of Education

June 21, 2013CCSSO 2013 Presentation 4

Michigan currently has six assessment programs.◦ Michigan Educational Assessment Program

(MEAP)◦ Michigan Merit Examination (MME)◦ MEAP-Access (2%)◦ MI-Access (1%)◦ English Language Proficiency Assessment (ELPA)◦ Michigan’s Interim Assessment System

Five of these assessment programs use reading passages for the assessments.

June 21, 2013CCSSO 2013 Presentation 5

Similar to many states, Michigan was purchasing permissions for contexts for all programs.

Leadership made a decision to charge the test development team with transitioning to 100% commissioned passages for all programs.

Michigan wanted to look to authors from Michigan to write pieces for Michigan’s assessments.

June 21, 2013CCSSO 2013 Presentation 6

Cost Ability to modify the pieces at the time of

item writing and item review if necessary to produce high quality items and the required quantity of items

Ownership of the contexts (reduced effort in continually renewing or tracking copyright permissions)

Reduction in information transfer headaches during vendor changes!

June 21, 2013CCSSO 2013 Presentation 7

Michigan has its own item banking and authoring system (IBS).◦ Includes a passage writing and reviewing module

Authors may work online in the IBS on-site where training takes place, or they may work elsewhere.

Authors may work in a word-processing system of their choice and then enter their work into the IBS.

Flexibility for the authors is key!◦ The reading contexts are automatically in the

same system in which items are written and tests are built.

June 21, 2013CCSSO 2013 Presentation 8

Reduces information migration between various systems to get the context to the item-writing phase

Supports Michigan children’s authors Provides for flexibility with the contexts,

allowing Michigan to generate first-rate assessments for students

Reduces permissions cost and time

June 21, 2013CCSSO 2013 Presentation 9

Authors took some time to learn the ins and outs of writing for assessments just as item writers do. (“Why can’t we write about certain topics?”)

We learned how to create context-writing training and work sessions differently than we would for item writing (flexibility, space, open “check-in” opportunities).

June 21, 2013CCSSO 2013 Presentation 10

—Kara Courtney

June 21, 2013CCSSO 2013 Presentation 11

List provided by Michigan Reading Association through the Michigan Department of Education contact◦ Letter of interest◦ Communications

13 published authors out of 23 committed to attend training

Over 60 quality contexts written within one month

June 21, 2013CCSSO 2013 Presentation 12

Writing for assessments Context specifications

◦ Word counts◦ Readability programs and other resources◦ Content concerns

Bias and sensitivity training◦ Topics to avoid◦ Socioeconomic issues◦ Other sensitivity issues

Technology training◦ Web-based program

June 21, 2013CCSSO 2013 Presentation 13

Michigan Item Bank System◦ Context authoring ability◦ External access

Benefits Challenges

June 21, 2013CCSSO 2013 Presentation 14

Provides rich text for students Ownership of context Relationships built with the authors Cost-effective approach to passage

development Generation of topics for all Michigan

assessment programs

June 21, 2013CCSSO 2013 Presentation 15

Published authors and assessments Working relationships Expanding the pool

June 21, 2013CCSSO 2013 Presentation 16

—Michael P. Spradlin

June 21, 2013CCSSO 2013 Presentation 17

Educators/administrators are unaware of the authorial talent that resides in each state.

June 21, 2013CCSSO 2013 Presentation 18

June 21, 2013CCSSO 2013 Presentation 19

Who better to write about Michigan than authors from Michigan?

Authors work at honing their ability to engage readers. Writing for assessments seems to be a natural progression.

Economic standpoint—keep money in Michigan.

June 21, 2013CCSSO 2013 Presentation 20

Usually work within guidelines, restrictions based on age of readers, reading levels, etc.

Authors are used to being edited. Not all is new, but in some regards writing

for assessments can be a challenge. Working one-on-one with DRC and MDE staff

June 21, 2013CCSSO 2013 Presentation 21

On-site training Individualized training for each assessment

area (MEAP, ELPA, Access all have some similarities and differences.)

Ability to dedicate time and effort to the specific program

Receive immediate feedback Writing two to four pieces a week Assignments can be completed at home

June 21, 2013CCSSO 2013 Presentation 22

Digging into nonfiction topics Bringing forward unknown or unique stories

about Michigan and creating topics around them to engage readers

June 21, 2013CCSSO 2013 Presentation 23

—Shirley Neitzel

June 21, 2013CCSSO 2013 Presentation 24

Published writers connect with their readers.

June 21, 2013CCSSO 2013 Presentation 25

June 21, 2013CCSSO 2013 Presentation 26

Local authors are “repository for arcane information”

Authors want students to do well on assessment tests

Authors appreciate the working relationship with DRC and MDE

June 21, 2013CCSSO 2013 Presentation 27

Training sessions are collegial

Assignments are specific

Topics are mutually agreed to before writing begins

Interesting, challenging work

Flexibility

June 21, 2013CCSSO 2013 Presentation 28

On-site training

Prompt feedback

Off-site work

June 21, 2013CCSSO 2013 Presentation 29

Supplemental income

Satisfaction of making a contribution to the education of students

June 21, 2013CCSSO 2013 Presentation 30

—Patty McDivitt

June 21, 2013CCSSO 2013 Presentation 31

Benefits◦ Developers can control for such considerations

as text complexity, fairness, sensitivity, and freedom from bias, quality of writing

◦ Develops passages that will truly lend themselves well to items

Key drivers for success◦ Training◦ Item banking/authoring system◦ Flexibility for writers

June 21, 2013CCSSO 2013 Presentation 32

With careful planning and commitment, the goal of using the work of published authors can be achieved.

June 21, 2013CCSSO 2013 Presentation 33

June 21, 2013CCSSO 2013 Presentation 34