© amanda vanderheyden, do not reproduce without permission using rti to advance learning in...

© Amanda VanDerHeyden, Do Not Reproduce Without Permission

Using RtI to Advance Learning in Mathematics

Amanda VanDerHeyden

Education Research and Consulting, Inc.

7 Years

Highly effective teachers show gain of 1.5 grade equivalents. Ineffective teachers show gains of .5 grade equivalents. These gains are independent of other risk factors associated with demographics. 2

Subtle Tension• Schools want

– scores to increase– vulnerable children to

show gains– to avoid negative AYP

labels– parents/communities

to have confidence in schools

– all children to have access

– meet legal obligations

• Families and Caregivers want their children– to be happy at school– to learn skills that will

help them succeed in college and beyond

– to develop life-long love for learning

– to grow into well-rounded, independent citizens

3

What do Families Want?• Improved learning• Transparent decisions• Active system problem-

solving• Efficient use of

resources• What was my child’s

score? What did you do differently? What effect did it have? What are we doing next?

4

Fool’s Gold

• If you are poor, of minority ethnicity, or a boy, you have a much higher probability of going to special ed and a much higher risk of academic failure.

• Special education placement does not improve outcomes for kids in the high-incidence categories and, in fact, is associated with risk.

5

Grade level corresponding to age

1 2 3 4

Re

ad

ing

gra

de

lev

el

4

3

2

1

5

2.5

5.2

At Risk on Early Screening

Early Screening Identifies Children At Risk of Reading Difficulty

Low Risk on Early Screening

This Slide from Reading First Experts

J

From Reading First6

Grade level corresponding to age

1 2 3 4

Re

ad

ing

gra

de

lev

el

4

3

2

1

5

2.5

5.2

Early Intervention Changes Reading Outcomes

At Risk on Early Screening

Low Risk on Early Screening

3.2

Control

With research-based core but without extra instructional intervention

4.9

Interventio

n

With substantial instructional intervention

This Slide from Reading First Experts

J

From Reading First7

New Assumptions with RtI

• Most children should successfully respond to intervention.

• Most children in a class should score at benchmark levels given adequate instruction.

• Intervention failure should be a rare event. Where it is not rare, implementation error should be the first suspect.


• Instructing without assessment or intervening without assessment data is akin to driving without a map.

• With data, any solution becomes a hypothesis to be tested.

• We need to focus more on supporting solution implementation and evaluating solutions to be sure they work.

• Effective teachers, administrators, and schools are defined by the results they produce.


What does RtI Mean for your Child?

• High-performing?– Use data to enrich and challenge, smarter

allocation of resources means more available for enrichment

– Children ready for advanced coursework• Average student?

– Children ready for advanced coursework• Low performing?

– Accelerated growth, reduction of risk for failure, mastery of essential skills


Objectives Today

• Understand how to lead excellent implementation of MTSS/RtI for mathematics

Data = Fuel

• To determine risk• To evaluate systemic problems• To plan instructional changes system-wide• To plan intervention for individual, small

groups, or whole classes as supplement to core

• To evaluate intervention effects and inform referral decisions

Multi-Tiered Academic Interventions

Tier I: Universal screening and progress monitoring with quality core curriculum: All students,

Tier II: Standardized interventions with small groups in general education: 15% to 20% of students at any time

Tier III: Individualized interventions with in-depth problem analysis in general education : 5% of students at any time

Tier I

Types of Math Knowledge

• Conceptual - the understanding that math involves an interrelated hierarchical network that underlies all math-related tasks

• Procedural - the organization of conceptual knowledge into action to actually perform a mathematical task (Hiebert & Lefevre, 1986).

• Which comes first?– Sequence may be specific to the domain or the

individual (Rittle-Johnson & Siegler, 1998; Rittle-Johnson, Siegler, & Wagner, 2001)

– But the two are clearly interrelated.


Challenges• Ratio of 6:1 • NAEP data show improvements but not for

ethnic minorities and low SES students• Lack of streamlined resources• Insufficient instructional time allocated to

mathematics• Math proficiency related to income post-

graduation, success in college• Students who are not proficient and enroll in

remedial classes post-secondary are less likely to graduate


Slavin & Lake (2008)

• 87/256 reviewed studies met rigorous inclusion criteria

• 13 categorized as examining curricula

• 36 categorized as computer-assisted instruction

• 36 categorized as instructional process

+ 0.10

+ 0.19

+ 0.33


Conclusion

• If you want to change math learning outcomes, you have to change the quality of the instructional interaction between student and teacher

• So what are the characteristics of quality core instruction in mathematics?


Begin with Number Sense• Numbers represent quantity and have

magnitude• One number may be bigger than another

number or quantity• Numbers have a fixed order with numbers later

in the sequence representing greater quantities

– Begins with counting in sequence, counting objects, comparing quantities, adding and subtracting numbers. Leads to understanding of associative, commutative, and distributive property and place value.

Griffin (2004)


Integrate Instruction for• Procedural and operational fluency with

conceptual understanding• e.g., emergence of the “count-on” strategy as

children’s understanding of ordinality and associative property develop– Estimate, discuss solutions, verify solutions, practice

application


Sequence Skills Logically and Provide Adequate Instructional Time• “a mile wide and an inch deep”• Make tough decisions about which skills are

essential and ensure mastery of those skills• NMP says

– whole number add/sub by grade 3– mult/div by grade 5– Operations with fractions, decimals, percentages– Operations with pos/neg integers– Operations with pos/neg fractions– Solving percentages, ratios, and rates to balance

equations


Use Data to Fuel DecisionsFrom VanDerHeyden (2009)


Match Instructional Strategy to Learner Competence

Increase Discriminability/ Stimulus Control

Ensure 100% correct responding

Match Instruction to Learner Competence

Opportunities to Respond; Practice to Mastery

Build Fluency

Increased range of stimuli

Response Variation- Build response set

Improve Maintenance

Student Competence

Goal of Intervention

Intervention Example

Acquisition Task Establish 100% correct

Cover, copy, and compare

Independent Task

Build fluency Flashcards, timed performance with incentives, response cards

Mastery Task Establish robust application

Guided practice intervention

What is Balanced Math Instruction?

Math Proficiency

Ensure acquisition of key concepts in math

Build conceptual understanding to

fluency

Provide opportunities to generalize skills to

novel problems

Opportunities to predict, estimate, verify, and discuss

solutions

Common Core Content Standards

• Streamlined• “Asking a student to understand

something means asking a teacher to assess whether the child has understood it.”

• Hallmark of understanding: student can explain why a mathematical statement is true or where a rule comes from.


Roadmap to Lesson Planning

• Do students understand? Can they do it?• How will you

– Establish conceptual understanding?– Build fluency?– Provide applied practice and discussion?



Tier 1: Screening

• 3 times per year• More frequently if problems are detected• Probably two probes required• Computation probes work well-- consider

state standards• Math Screening

• 2 minutes. Scored for Digits Correct per 2 min


Class-wide Screening


Feedback to Teachers


Tier 1 or 2: Class-wide Intervention


No Class-wide Problem Detected


Tier 2: Can’t Do/Won’t Do Assessment

• “Can’t Do/Won’t Do”• Individually-administered• Materials

– Academic material that student performed poorly during class assessment.

– Treasure chest: plastic box filled with tangible items.

3-7 minutes per child


Decision Rule Following Can’t Do/Won’t Do Assessment


Tier 3: Individual Intervention

• Conducted by classroom teacher• Protocol based• Follows adequate functional assessment


Response to Intervention

Before Intervention During Intervention

Avg. for his Class

Intervention in Reading

#Correct

Intervention Sessions

Each Dot is one Day of Intervention


Before Intervention During Intervention

#Correct

Avg. for his Class

Response to Intervention


Using Screening Data to Identify Class-wide and

System-wide Instructional Problems

Step 1: Identify the need for Tier 1 or 2 Intervention


Screening tells you

• How is the core instruction working?• What problems might exist that could be

addressed?• Most bang-for-the-buck activity• Next most high-yield activity is classwide

intervention at Tier 2.


Consider

• The Task• Integrity of Administration• Reliability of Scoring• Use software to organize the data

Mult 0-9 4th Grade Fall Screening

Start with a Helicopter View

45

Second Grade Math

46

Third Grade Math

47

Where system problems are detected, deploy system interventions and: Verify Rapid Growth in all Classes

48

Look for Lagging Classes– and Respond

49

Teac

her

1

Teac

her

2

Teac

her

3

Teac

her

4

Teac

her

5

Teac

her

6

Teac

her

7

Teac

her

8

Teac

her

9

Teac

her

10 Teac

her

11 Teac

her

12

50

Set System Goals- Track- And Respond

1 2 3 4 5 6 7 80

10

20

30

40

50

60

70

80

90

First Graders N = 250

Poverty

Linear (Poverty)

Not in Poverty

Linear (Not in Poverty)

Weeks

wo

rds

read

co

rrec

tly

per

min

ute

51

How Can MTSS Help?• Organize small groups based on student

proficiency (acquisition, fluency, generalization)

• Use Classwide intervention to build fluency in pre-requisite skills (I’ll explain)

• Use intensive, individualized interventions to conduct acquisition interventions following functional academic assessment (I’ll show you how)

• Use screening data to connect instructional strategies to student proficiency


© Amanda VanDerHeyden, Do Not Reproduce without Written Permission

Gradewide Problem?

No

Classwide Problem?

Yes

Intervention

No

Yes

School-Wide Problem?• Examine core instruction materials and

procedures– Instructional time– Research-supported curric materials– Calendar of instruction– Understanding and measurement of mastery

of specific learning objectives


• Establish priorities for improvement and determine timeline

• Add a supplemental instructional program with weekly PM

• Examine and respond to implementation effects each month. Share w/ feeder pattern & connect to long-term effects.


School-Wide Problem?

Fall Winter Spring0

10

20

30

40

50

60

70

80

90

Percentage of Students At Risk

Percentage of Students At Risk


Teac

her

1

Teac

her

2

Teac

her

3

Teac

her

4

Teac

her

5

Teac

her

6

Teac

her

7

Teac

her

8

Teac

her

9

Teac

her

10 Teac

her

11 Teac

her

12

58

• Demographics should become more proportionate in failure or risk groups over time.

• Percentage of students “on track” should improve (look at percent enrolling in and passing algebra, AP enrollments and scores, Percent taking and meeting ACT benchmarks).


Grade-wide Problem?

• Examine core instruction procedures• Begin class-wide supplement and PM

weekly• Conduct vertical teaming with preceding

and subsequent grade levels to identify strategies to ensure children attain grade-level expected skills in future.


Teac

her

1

Teac

her

2

Teac

her

3

Teac

her

4

Teac

her

5

Teac

her

6

Teac

her

7

Teac

her

8

Teac

her

9

Teac

her

10 Teac

her

11 Teac

her

12

61

Small Group Problem

• Use Tier 2 time to provide more explicit instruction following standard protocol.

• Monitor weekly. Exit students based on post-intervention performance not in the risk range on lesson objectives and screening criterion.

• When most children are responding well, identify children for Tier 3.


• About 90% of children should respond successfully to Tier 2 intervention

• Successful responders should surpass screening criterion at higher rates on subsequent screenings.

• Successful responders should pass high-stakes at higher rates than before use of Tier 2 strategies.


Individual Problem?

• Conduct individual assessment to establish targets, identify effective intervention, and specify baseline.

• Prepare all materials• Monitor weekly and troubleshoot to

accelerate growth


• Most children participating in Tier 3 should respond successfully. More than 5% of screened pop is a red flag.

• Focus on integrity of intervention.• Growth should be detectable within two

weeks.• Troubleshoot interventions that aren’t

working.


• Successful responders to Tier 3 should fall into risk range on subsequent screenings at lower rates.

• Successful responders should pass high-stakes at higher rates.

• Unsuccessful responders should qualify for more intensive instruction at higher rates.

• Responders/nonresponder should be proportionate by demographics.


Let’s Talk about Two Pitfalls

• Loosely Defined Model• Over-assessment

67

Your Model is “Too Loose” If

• Results are inconsistent across schools and/or over time

• There are long delays between decisions• There are cases without a final decision

68

Assess Smarter

• First, select the best measures and understand what the “hit” rate is

• No measure is perfect and adding more measures may not (most likely will not) increase the “hit” rate

• What do I mean by a “hit” rate?

69

“Hit” Rates Summarize Accuracy of Decisions

70

Users Must Weigh• The costs of false positive errors and false

negative errors for each decision.– For Screening Decisions – A priority is placed

on avoiding false negative errors typically.– As a result, many screening systems burden

systems with high false-positive error rates. – High error rates cause users to lose

momentum and can attenuate intervention effects systemwide.

– Collecting “more data” does not necessarily improve the hit rate.

72

Schools are Drowning in Data and the Same Children Still

Can’t Read (or Count)

• Are we making a difference?• Are we changing the odds?

73

Take an Assessment Inventory

74

Verify Screening Adequacy

75

Exploit Existing Data and Respond- First, Verify Core

76

Decision “Hit Rates” Can be Examined to know if

• Use of an assessment or intervention improves outcomes over time (increases the odds of student success)

• You can compute the probability of passing or failing the high-stakes test if a student has passed or failed a screener (called the post-test probability)

• e.g., VanDerHeyden, A. M. (2010). Determining early mathematical risk: Ideas for extending the research. Invited commentary in School Psychology Review, 39, 196-202.

82

Bad Decisions are Not Benign

• Parent/School Bonding

• Community Support

• Play• Rest

• Field Trips• Special

Projects

• Art• Music

Literacy Mathematics

Social SkillsLanguage and Writing

83


How-To Classwide Math


Intervention Plan- 15 Min per Day• Protocol-based classwide peer tutoring,

randomized integrity checks by direct observation

• Model, Guide Practice, Independent Timed Practice with delayed error correction

• Group performance contingency• Teachers encouraged to

– Scan papers for high error rates– Do 5-min re-teach for those with high-error rates– Provide applied practice using mastery-level

computational skill


• Usually the higher-performing student, goes (models) first.• Rotating high performers helps maintain motivation


Materials Needed

• Computer and software to organize data• Student data imported. Clerical person to enter data on-

site for tier 1 screen only.• Color printer to print graphs + extra color cartridges• Probe materials, digital count-down timers• Intervention protocols, intervention materials (e.g.,

flashcard sets, reading materials)• Access to copier and some assistance with copying• Reinforcers for treasure chest (no more than $500 per

school)


Measurement Plan

• Weekly probe of Intervention skill• Weekly probe of Retention of previously

mastered computational skills• Monthly probe using GOM approach to

monitor progress toward year-end computational goals

• To this you might add an application measure


Sample Sequence

Kindergarten, 1st Semester


Kindergarten, 2nd Semester



Intervention Plan

• Class Median reaches mastery range for skill, next skill is introduced

• Following promising results at one site in 2002-2003, lead to implementation district-wide grades 1-8 for all children by 2004-2005.

Instructional Criteria• MATH

– K: • 0-7 Count Objects, Circle Number• 0-5 Count Objects, Write Number• 0-4 Identify Number, Draw Circles• 0-5 Rapid Discrimination (sorting)

– Grades 1-3• 0-19 dc/2 min Frustration• 20-39 dc/2 min Instructional• 40+ dc/2 min Mastery

– Grades 4-6• 0-39 dc/2 min Frustration• 40-79 dc/2 min Instructional• 80+ dc/2 min Mastery

AcquisitionChild response is inaccurate

Fluency

Child response is accurate but slow

Generalization Child response is fluent

Salient cues, frequent & high-level prompting, immediate feedback, more elaborate feedback, sufficient exemplars of correct/incorrect, controlled task presentation.

Intervals of practice, opportunities to respond, delayed feedback, ensure reinforcement for more fluent performance.

Cues to generalize, corrective feedback for application and problem-solving, systematic task variation, fading of support.

A More Powerful Way to Define Intervention Intensity


Class-wide Math Intervention


Decision making

• Review data to make decisions:

DATA OUTCOME 1: Class median is below mastery range and most students gaining digits correct per week.

ACTION: Consider implementing intervention for an additional week and then review progress again.


Decision making DATA OUTCOME 2: Class median is below

mastery range and most students are not gaining digits correct per week:

ACTION: Check Integrity first and address with training if needed. Consider implementing intervention for an additional week with incentives or easier task and then review progress again.


Decision making

DATA OUTCOME 3: If the class median is above mastery range then consider:

ACTION: Increasing task difficulty and continuing classwide intervention.

ACTION: For students performing 1 SD below the class mean, consider Tier 3.


Results


Tier 1 Screening Indicates Class-wide Problem


Tier 2: Class-wide Intervention

Teacher F Mult 0-12

0

20

40

60

80

100

120

10/2

4/20

03

10/3

1/20

03

11/7

/200

3

11/1

4/20

03

11/1

8/20

03

Weeks

Dig

its

Co

rrec

t Tw

o M

inu

tes


Increased Difficulty- Intervention Continues


Instructional range

Frustrational range

Pre-post changes to performance detected by CBM

Each bar is a student’s performance


Fourth Grade


Computation Gains Generalized to High Stakes TestImprovements

(Gains within Multiple Baselineshown as pre-post data)


Gains within Multiple Baseline (shown as pre-post data)


District-Wide RCT 4th & 5th Graders


treatment

control

• Effects on year-end scores significant at fourth grade. Effects strongest for students who were lowest performing on the prior year’s test score.

• CBMS showed strong effects, both grades.

• Integrity varied by class and variations explained effects

116

Overall

117

For Vulnerable Students

118

For Vulnerable Students

119

Conclusions

• Low-performing students more prone to have week(s) of missing data.

• Probability of failure was reduced at a greater rate for students who receive free and reduced lunch, students receiving special education, and for African American students.

120

Let’s Talk about Another Pitfall

• Overemphasizing intervention selection and under-emphasizing intervention management

121


___% of interventions are not used

without support

80


Common Pitfalls

• Most interventions fail because they are not implemented correctly

• Standard protocol interventions facilitate accurate implementation and can work

• Too much time is spent on problem admiration and dissection


Troubleshoot Intervention Support Yes NoWas the intervention developed to ensure that it required minimal classroom time and resources and fit within daily classroom routines?

Are materials readily available to the teacher?

Was a step-by-step “coach card” provided?

Was the teacher shown how to implement the intervention by a “coach?”

Did the coach observe implementation of the intervention to ensure that the teacher could use the intervention correctly and had all needed materials?

Was weekly follow-up support provided to the teacher after initial training?

Are integrity data graphed to show used correctly?

Is an administrator involved?From Witt, VanDerHeyden, & Gilbertson, 2004

Integrity Failures are Sentinel Events

• Untreated integrity problems become student learning deficits, schoolwide learning problems, and false positive decision errors

• Integ problems affect dose and quality of the treatment (an intervention implemented with fidelity is a functionally different intervention than one implemented inconsistently

• Integ positively correlated with student learning gains, amount of intervention covered

• Even veteran sites require monitoring and follow-up

125

Sometimes it’s the Simple Things• Proximity to trainer• Child availability for intervention sessions• Intervention error (e.g., modeling too

rapidly, failing to give feedback)• Materials available• No one’s watching• Tracking and troubleshooting

implementation failures• Remember, intervention failures should be

rare 126

Just like your mama told you: INTEGRITY MATTERS

59% Integ 96% Integrity

127

128VanDerHeyden, McLaughlin, Algina, Snyder (in press). AERJ

To Avoid Pitfalls• Specify measures, decision rules, and

intervention management procedures• Obtain the best data• Obtain only the data necessary to make

accurate decisions at each stage• Plan system interventions where system

problems are detected• Actively manage intervention

implementation129

Ask• What are our system goals?• What data are we collecting to reflect

progress?• How are we responding to lack of progress

(how often, what resources)?• How do data inform professional

development decisions, text/material/resource adoptions, allocation of instructional time?

• How do data tie into personnel evaluation?130

Ask• Are we changing the odds of success in

our schools?• What are our special targets and priorities

(e.g., numeracy, high-mobility, etc.)• Are we operating as efficiently as

possible?• Are teachers adequately supported (i.e.,

someone responds to data and goes in to coach and support)?

• Do our instructional leaders follow data?

131

Avoid Common Mistakes• Exploit existing data to know if efforts are

working– % at risk fall, winter, spring by grade– % of class-wide problems fall, winter, spring by

grade– % of f/r lunch at risk should mirror % of f/r lunch

overall, same for ethnicity and sp ed– Reduced risk across grades– Decreased evaluations, proportionate, & accurate

• Specify what you are going to do about it• Implement solution well• Follow-up and respond to implementation

failures132

For More Information• Amanda VanDerHeyden

– [email protected]– 251-300-0690

• www.isteep.com and www.gosbr.net • www.rtinetwork.org• www.nasdse.org (blueprints)• Keeping RTI on Track: How to Identify, Repair and Prevent

Mistakes That Derail Implementation• http://www.shoplrp.com/product/p-300620.html• Or 1-800-341-7874• http://www.jeabjaba.org/abstracts/JabaAbstracts/26/26-597.Htm

(Fixsen & Blasé, 1993)• Hattie (2009). Visible Learning.

133

mailto:[email protected]

http://www.isteep.com/

http://www.gosbr.net/

http://www.rtinetwork.org/

http://www.nasdse.org/

http://www.shoplrp.com/product/p-300620.html

http://www.jeabjaba.org/abstracts/JabaAbstracts/26/26-597.Htm

© amanda vanderheyden, do not reproduce without permission using rti to advance learning in...

Documents

permission slide

reading grade level

early screening low

reduction of risk

risk factors

vulnerable children

intervention failure

grade equivalents