MN RtI Center1

Progress Monitoring in Reading:Why, What, and How

A module for pre-service and in-service professional development

MN RTI CenterAuthors: Lisa Habedank Stewart, PhD & Adam Christ, graduate

student Minnesota State University Moorhead

www.scred.k12.mn.us click on RTI Center

MN RtI Center2

MN RTI Center Training Modules

This module was developed with funding from the MN legislature It is part of a series of modules available from the MN RTI Center

for use in preservice and inservice training:

Module Title Authors

1. RTI Overview Kim Gibbons & Lisa Stewart

2. Measurement and RTI Overview Lisa Stewart

3. Curriculum Based Measurement and RTI Lisa Stewart

4. Universal Screening (Benchmarking): (Two parts)

What, Why and How

Using Screening Data

Lisa Stewart

5. Progress Monitoring: (Two parts)

What, Why and How

Using Progress Monitoring Data

Lisa Stewart & Adam Christ

6. Evidence-Based Practices Ann Casey

7. Problem Solving in RTI Kerry Bollman

8. Differentiated Instruction Peggy Ballard

9. Tiered Service Delivery and Instruction Wendy Robinson

10. Leadership and RTI Jane Thompson & Ann Casey

11. Family involvement and RTI Amy Reschly

12. Five Areas of Reading Kerry Bollman

13. Schoolwide Organization Kim Gibbons

MN RtI Center3

Overview

This module is Part 1 of 2 Part 1: Why, What, How to Progress Monitor

Why do it?

What do we mean by progress monitoring?

How do you do it?

Part 2: Using Progress Monitoring Data

MN RtI Center4

Is this student making progress?Adam Gr 4

0

10

20

30

40

50

60

70

80

90

100

110

120

130

140

week

wo

rds

corr

ect

pe

r m

in (

gra

de

leve

l pa

ssa

ge

)

wrc

errors

Repeated Reading 1:1, 20 min day Repeated Reading 1:1, 10 min 2xday

MN RtI Center5

Adapted from Logan City School District, 2002

Curriculum and Instruction Assessment

School Wide Organization &

Problem Solving Systems

(Teams, Process, etc)

Assessment: One of the Key Components in RTI

MN RtI Center6

Assessment and Response to Intervention (RTI)

A core feature of RTI is identifying a measurement system Screen large numbers of students

Identify students in need of additional intervention

Monitor students of concern more frequently 1 to 4x per month

Typically weekly Diagnostic testing used for instructional planning to

help target interventions as needed

MN RtI Center7

Screening Data and Progress Monitoring can be linked

The goal is to have a cohesive system.

If possible, use the same measures for both screening and progress monitoring (e.g, CBM).

Screen ALL students 3x per year (F, W, S)

Strategic Support and MonitoringStudents at Some Risk

Intensive Support & Monitoring for

Students at Extreme Risk

MN RtI Center8

Why Monitor Progress? When teachers USE progress monitoring

Students learn more! Teachers design better instructional

programs Teacher decision making

improves Students become more aware of

their performance Safer & Fleishman, 2005

MN RtI Center9

Why Monitor Progress?

“In God we trust…

All others must have data.”Dr. Stan Deno

MN RtI Center10

Why Monitor Progress?

We do NOT KNOW ahead of time whether an intervention will be successful for an individual student

Do they ASSUME in the hospital that your heart is working just fine after your bypass surgery? After all… the surgery works well for MOST patients…..

MN RtI CenterDRAFT May 27, 2009 11

To change what you are doing with a student if it is not working (formative assessment) so you are effective and efficient with your time and instruction

To help make decisions about instructional goals, materials, levels, and groups

To aid in communication with parents

To document progress for special education students as required for periodic and annual reviews

Credit: based on slide by Dr. Kim Gibbons, SCRED

Why Progress Monitor Frequently?

MN RtI Center12

Credit: SCRED

Change (and Keep Changing) if Instruction isn’t Working…

MN RtI Center13

You Can Also Use Monitoring Data to Celebrate Success!

MN RtI Center14

What is Progress Monitoring?

Standardized measures Reliable Valid

Tied to important educational outcomes

Given frequently (e.g., weekly) Simple, brief, efficient, and cheap Sensitive to growth over short periods of time

MN RtI Center

Measurement and RTI: Progress Monitoring

Reliability coefficients of r=.90+ Well documented treatment validity! Test and scores are very sensitive to increases or

decreases in student skills over time Evidence of what slope of progress (how much growth in a

day, week or a month) is typical under what conditions can greatly increase your ability to make decisions

VERY brief, easy to use, affordable, alternate forms, and results/reports are accessible immediately

MN RtI CenterDRAFT May 27, 2009 16

Standards for Scientifically Based Progress Monitoring

Reliability Quality of a Good Test

Validity Quality of a Good Test

Sufficient Number of Alternate Forms and of Equal Difficulty

Essential for Progress Monitoring

Evidence of Sensitivity to Intervention Effects Critical for Progress Monitoring

Benchmarks of Adequate Progress and Goal Setting

Critical for Progress Monitoring

Rates of Improvement are Specified Critical for Progress Monitoring

Evidence of Impact on Teacher Decision-Making Critical for Formative Evaluation

Evidence of Improved Instruction and Student Achievement

Gold Standard

National Center for Progress Monitoringwww.studentprogress.org

MN RtI Center17

Many tools that meet standards are

members of the Curriculum-Based

Measurement (CBM)“family.”

www.studentprogress.org click on “Tools”

MN RtI Center18

Buyer Beware!

Many tools may make claims about monitoring progress but…

Is it reliable and valid?

Is it tied to important educational outcomes?

Can it be given frequently (e.g., weekly)? Simple, brief, efficient, and cheap Sensitive to growth over short periods of time

MN RtI Center19

Short Term (Mastery) and Long Term

Progress Monitoring

Short TermMastery Monitoring

Test subskill mastery and individual lesson effectiveness

Ex: Q&A, worksheetsfollowing directions

unit tests, “hot” readsaccuracy, skills “checks”

CBE, cba

Long TermGeneral Outcome Measures

Test retention, generalization and progress

toward overall general outcome (reading)

Ex: CBM, DIBELS

MN RtI Center20

Both Mastery Monitoring and Long Term Progress Monitoring are Important

Sometimes mastering subskills doesn’t generalize to the general outcome or students don’t retain the information over time

For example: Melissa is very good at decoding letters and reading

individual words, but is not generalizing these skills to reading text with automaticity and comprehension.

Adam was really good at using his comprehension strategies and using those when they were working on these skills in class (showed mastery), but when they moved on to another unit he quit using the strategies.

MN RtI Center21

How Often Do I Need to Monitor Progress?

Informally we do this all the time! For small instructional adjustments (repeat

the lesson, how much help to give, etc.)

In a standardized way to make sure we are “on track” with this student?- depends on level of concern For students in reading who are behind already,

monitor progress toward generalized outcome 1 to 4x per month, ideally weekly

MN RtI Center22

How Do You Collect Frequent Progress Monitoring Data?

Which students?

What measures?

What materials?

How often?

Who collects the data? Where? When?

MN RtI Center23

Which students?

Students of concern

Below target

Getting “extra” intervention or help

Tier 2 or Tier 3 services

MN RtI Center24

What measures?

Web-based sources for information on measures as well as access to materials, web-based data management, etc. www.aimsweb.com dibels.uoregon.edu www.edcheckup.com

Disclaimer: More data management systems are being developed and marketed all the time to provide information, resources, and data management. This is just a sample of the types of web-based resources available. Be a critical consumer!

MN RtI Center25

What materials?

When possible, students are monitored using grade level materials

E.g., student reads a different grade level passage or “probe” each week

If this is not possible due to frustration “test down” and use the highest grade level of materials possible Periodically “check” how the student is doing on grade level

materials and move into grade level materials as soon as possible

MN RtI Center26

Example of Testing Down: Reading “Survey Level” Data for Gus, Gr 4

0

10

20

30

40

50

60

70

80

90

100

110

120

130

140

150

160

170

180

190

200

Grade 1 Grade 2 Grade 3 Grade 4 Grade 5 Grade 6

wor

ds c

orre

ct p

er m

inut

e

Note:

-Error rates high (5-15) in Gr 4,

-slightly lower (4-8) in Gr 3 & 2,

-and much lower (1-4) in Gr 1.

MN RtI Center27

What Material Should We Use…

To “instruct” Gus?

To monitor Gus’ progress?

MN RtI Center28

How Often Do We Monitor Progress?

Depends on the sensitivity of the measure and the level of concern we have about the student, but 1 to 4x per month typically

For CBM Oral Reading Fluency Weekly with 1 passage (this is most common) Every 3 weeks with 3 passages

MN RtI Center29

Who does it? When? Where?

Anyone trained in the procedures can collect progress monitoring data Classroom teachers, special education teachers,

Title teachers, aides, related services staff, adult volunteers

Be creative but careful

When and Where? At a time and place that will provide valid information Use common sense

MN RtI Center30

Sharing the Data

Just having progress monitoring data is not enough. You need to USE it.

Scheduled graph review dates

Grade level meetings

Problem solving meetings

MN RtI Center31

Using Progress Monitoring Data: Is this intervention working?

Adam Gr 4

0102030405060708090

100110120130140

w eek

wor

ds c

orre

ct p

er m

in (g

rade

leve

l pas

sage

)

w rc

errors

See MN RTI Center Module 2 for information on graphs, decision rules, etc

MN RtI Center32

Remember: Garbage IN…. Garbage OUT….

Make sure your data have integrity or they won’t be good fer nuthin… Training Integrity checks/refreshers Well chosen measures and

materials

MN RtI Center33

Avoid Common Mistakes

Don’t use the same passage/probe every week!

Have an organized system in place Progress monitoring schedule for students Preprinted passages/probes in a binder An easy way to graph and look at the data Scheduled time to share/look at the data

MN RtI Center34

Remember: When teachers USE progress monitoring

Students learn more! Teachers design better instructional

programs Teacher decision making

improves Students become more aware of

their performance Safer & Fleishman, 2005

MN RtI Center35

Web Resources Research Institute on Progress Monitoring

http://progressmonitoring.net/ Includes…

A Study Group Content Module with 15 sections on CBM including activities http://progressmonitoring.org/pdf/cbmMOD1.pdf

Progress Monitoring Leadership Team Content Module with 6 sections (e.g. measureable goals, decision making) including activities http://progressmonitoring.org/pdf/cbmMODldrshp.pdf

Handouts, videos, and power point presentations Technical reports of CBM measures

MN RtI CenterDRAFT May 27, 2009 36

Web Resources, Cont’d www.studentprogress.org

http://www.studentprogress.org/chart/chart.asp

http://www.rti4success.org/ click on Progress monitoring on right side

www.interventioncentral.org look for information on CBM, graphing, etc.

www.aimsweb.com, www.edcheckup.com, dibels.uoregon.edu Look for information about progress monitoring as well as

access to materials and graphing for progress monitoring

MN RtI Center37

Recommended Texts

Riley-Tillman & Burns. (2009). Evaluating Educational Interventions. Guilford Press.

MN RtI Center38

Print Resources available with this module

Safer & Fleishman. (2005). How student progress monitoring improves instruction, Educational Leadership, 62(5), 81-83.

Fuchs. Progress monitoring within a multi-level prevention system. Retrieved June 5, 2009, from RTI Action Network Web site: http://www.rtinetwork.org/Essential/Assessment/Progress/ar/MultilevelPrevention

Fuchs & Fuchs What is scientifically-based research on progress monitoring? From the National Center on Student Progress Monitoring (studentprogress.org). Retrieved June 14, 2009,

Jenkins, Hudson, & Hee Lee. Using CBM-Reading assessments to monitor progress. Retrieved June 5, 2009, from RTI Action Network Web site: http://www.rtinetwork.org/Essential/Assessment/Progress/ar/Using CBM/1

MN RtI Center39

Activity for Teachers or Practicum Students Obtain progress monitoring probes and graphs

Passages and graphing materials self-created or downloaded www.interventioncentral.org dibels.uoregon.edu

Sign up for an account with AIMSweb (instructor accounts and student accounts available) www.aimsweb.org

Practice administration and scoring Progress Monitor a “real” kid (ideally 2-4 kids of varying risk

levels monitored for at least 7-10 weeks) Graph, analyze, and use data

MN RtI CenterDRAFT May 27, 2009 40

Quiz

1.) What are at least four reasons why teachers should monitor progress?

2) What is the difference between mastery monitoring and frequency progress monitoring toward a general outcome?

3) How often should you monitor progress for students receiving extra help ?

MN RtI Center41

Quiz (Cont’d)

4.) When possible, students are monitored using… A.) above grade level materials. B.) grade level materials. C.) below grade level materials.

5.) Who can collect progress monitoring data?

MN RtI Center42

Quiz (Cont’d)

6.) To have integrity, your data collection must include what? A.) trained data collectors B.) integrity checks/refreshers C.) well chosen measures and materials D.) all of the above

MN RtI Center

Quiz (Cont’d)

7.) True or False? The most important use of frequent progress monitoring is to aid in communication with parents.

MN RtI Center

Note: The MN RTI Center does not endorse any particular product. Examples used are for instructional purposes only.

Special Thanks: Thank you to Dr. Ann Casey, director of the MN RTI Center, for

her leadership Thank you to Aimee Hochstein, Kristen Bouwman, and Nathan

Rowe, Minnesota State University Moorhead graduate students, for editing, writing quizzes, and enhancing the quality of these training materials