On the Horizon

Board of Education PresentationJanuary 9, 2012

Lynn K. McMullin

OBJECTIVES

Introduction

Changes in Testing on the Horizon

Impact

CHANGE is imminent…

FACT: This year’s kindergarteners will never take the CMT!

FACT: SBCAT (?) will soon be as well known to you as CMT

OVERVIEW of Changes

In June 2010, 31 states, including Connecticut, joined to

form the SMARTER Balanced Assessment Consortium (SBAC)

Submitted application in the ‘Race to the Top’ Assessment

Competition

OVERVIEW

The USDE awarded two

‘Comprehensive Assessment

Systems’ grants in September

2010

• One to SBAC• Other to the

Partnership for Assessment of Readiness for College and Careers (PARCC)

PARCC

» Through-course Assessments #1 and #2 ELA and Math; 1 – 3 tasks in a class period

» Through-course Assessment #3 ELA and Math; taken over several sessions or classes

» Through-course Assessment #4 ELA Speaking and Listening; each student presents;

teacher scores» End-of-Year Assessment

On Computer 45 – 60 questions

» Field testing 2012 – 2014 – Operational 2014

Report

scores in 2

weeks

OVERVIEW» CT is a GOVERNING state in SBAC

Computer-Adaptive Testing: CAT

An Adaptive Test is a test that dynamically

adjusts to the trait level of each examinee as the

test is being administered.

• Get it right, it moves you up

• Get it wrong, it moves you down

• Makes continuous adjustments as you work

Computer-Adaptive Testing: CAT

Developed in France around 1905, Alfred Binet’s IQ Test

Is still used in schools today

Is the standard against which IQ tests are compared

Incorporates all the elements of an adaptive test.

A different starting point can be used for each child

It uses an adaptive item-selection procedure.

Scoring method allows a common score to be

obtained from different subsets of items.

Length can vary by child with the use of a variable

termination rule.

Based on a 43,000 calibrated

item bank

Computer-Adaptive Testing: CAT

A starting rule for selecting the first item

A procedure for scoring item responses and

estimating trait level

A method of selecting the next item

A rule for ending the test

Pre-calibrated 43,000 item bank

Computer-Adaptive Testing: CAT

Range of item complexity…

Mike is using cubes that measure ¼ inch on each side to fill a box that has

a height of 5 ¼ inches, width of 3 inches, and length of 2 ½ inches. How many ¼ inch cubes will Mike need to

fill the box?

Mike is using cubes that measure ½ inch on each side to fill a box that has a height of 5

¼ inches, width of 3 inches, and length of 2 ½ inches. How many ½ inch cubes will Mike

need to fill the box?

Mike is using cubes that measure ½ inch on each side to fill a box that has a height

of 5 ½ inches, width of 3 inches, and length of 2 ½ inches. How many ½ inch

cubes will Mike need to fill the box?

ADVANTAGES OF A CAT

CAT equalizes the psychological environment of the test across all ability levels.• High-ability students will get about 50% of the

questions correct.• Low-ability students will get about 50% of the

questions correct.

ADVANTAGES OF A CAT

Efficiency: CATs are more efficient than conventional tests—they generally reduce test length by 50% or more.

Precision: A properly designed CAT can measure all examinees with the same

degree of precision.

Reporting: More accurate placement of students who previously scored

‘advanced’ and ‘below basic.’

ADVANTAGES OF A CAT

Reporting: Results can be made available more quickly (computer-based)

Test Security/Item Exposure: Students are presented with different test items

More Flexibility for Computer Capacity: Students do not need to be assessed on

the same schedule

ADVANTAGES OF A CAT

Strategies will change…

Students cannot change an answer to an item once they have submitted it.

Test prep will need to include this.

Because CAT is dynamic, it can recover from an occasional student

error in answering an item.

Literature shows little or no gain from answer changing.

ITEMS will change…

Animations, simulations, on-line access to information, video or audio

stimulus, moveable modelsTest prep will need to include this.

Elicit a response from the student (e.g., selecting one or more points on a graphic, dragging and dropping a

graphic from one location to another, manipulating a graph)

Innovative items are coming…

Artificial Intelligence (AI)

All constructed response items in the CAT will be AI scored

Items not scored with AI delivered outside of the CAT ‘engine’ (e.g., some elements of

performance tasks)

SBAC will require 10 - 20% read behind to ensure accuracy

AI scoring is nearly 100% reliable

SBAC Testing

Summative Assessment

(CAT)

Mandatory comprehensive assessment in

grades 3–8 and 11

Testing window within

the last 12 weeks of the instructional

year

Selected response,

short constructed response, extended

constructed response,

technology enhanced,

and performance

tasks

Snapshot

Accountability

High-level analyses of tr

ends

District & State

SBAC Testing

Interim Assessment

(CA)Optional content-cluster

assessment

Learning progressions

Available throughout

the year

Selected response,

short constructed response, extended

constructed response,

technology enhanced,

and performance

tasks

Actionable Feedback

Progress

Monitoring

Teacher & School Comparisons

New MODEL

The teacher shares the learning goals with

students and provides opportunities for

students to monitor their ongoing progress.

New Curriculum: Common Core State Standards

Importance of focusing time, energy and

resources on implementing the CCSS beginning this coming school year: Teachers

must read the standards

Mathematics

FOCUS, FOCUS, FOCUS – Deeper understanding of fewer concepts

COHERENCE – One year builds to next

FLUENCY – Standards expect speed and accuracy

DEEP UNDERSTANDING – fewer standards allow for this

APPLICATION – ability to apply what they know

SB…CAT (?) Testing

Spring 2011 – 2012 – Smarter Balanced will pilot SBCAT with some schools’ -- email to Superintendents asking about participation **

Spring 2012 – 2013 – Larger SBCAT pilot

Spring 2013 – 2014 – Every district will be required to pilot a portion of the SBCAT test

CT has applied for a CMT/CAPT moratorium for 2013-2014

Significant changes

12 week window for testing – to meet 1-to-1 computer requirements

End-of-year window – to allow for interim assessments

Less time required; more precise

43,000 test item bank

Significant changes

Innovative test items to match ‘real world’ applications• You wouldn’t really use a protractor on a

computer, rotate it, etc.• You really would click on a word to get its

definition or hear it pronounced

Significant changes

Text Selection

Text Complexity

Range and Quality of Texts

High-Quality Text-Dependent Questions and Tasks

Students’ Ability To Read Complex Texts Independently

Academic Vocabulary

Students encounter appropriately complex texts at

each grade level to develop skills and the conceptual knowledge they need for success in school and life.

Includes extensive work in class with texts that are at or

above grade level.

Includes short tests that require close reading;

50% literary, 50% informational (K – 6)

50% literary non-fiction (7-12)

80 – 90% of all questions should be text dependent questions which require

close reading, vs. skimming.

Words that readers will find in all types of

complex texts from different disciplines.

Now

• Join the limited pilot in 2012, if possible• Examine existing CAT’s

Now

• Study and align to the CCSS• Develop new instructional strategies

Now

• Plan for a 1-to-1 solution by 2015• Define a new set of test-taking strategies

Next Steps ??