national center on response to intervention rti implementer series module 3: multi-level prevention...
Post on 22-Dec-2015
218 Views
Preview:
TRANSCRIPT
National Center on Response to Intervention
National Center on Response to Intervention
RTI Implementer SeriesModule 3: Multi-Level
Prevention System
National Center on Response to Intervention
Session Agenda Welcome and Introductions Review What Is Multi-Level Instruction? Selecting Evidence-Based Practices Wrap-Up: Review, Questions, and Resources
2
National Center on Response to Intervention
Upon Completion Participants Will Be Able To: Use screening and progress monitoring data to make
decisions at all levels of the multi-level prevention system, including movement between levels.
Develop a multi-level prevention system. Select evidence-based interventions and practices.
3
National Center on Response to Intervention
REVIEW: SCREENING AND PROGRESS MONITORING
4
National Center on Response to Intervention 5
Defining RTI Response to intervention (RTI) integrates assessment
and intervention within a school-wide, multi level ‑prevention system to maximize student achievement and reduce behavior problems.
National Center on Response to Intervention 6
Defining RTI With RTI, schools identify students at risk for poor
learning outcomes, monitor student progress, provide evidence-based interventions and adjust the intensity and nature of those interventions based on a student’s responsiveness; and
RTI may be used as part of the determination process for identifying students with specific learning disabilities or other disabilities (in accordance with state law).
National Center on Response to Intervention 7
RTI as a Preventive Framework RTI is a multi-level instructional framework aimed at
improving outcomes for ALL students. RTI is preventive and provides immediate support to
students who are at risk for poor learning outcomes. RTI may be a component of a comprehensive
evaluation for students with learning disabilities.
National Center on Response to Intervention 8
Essential Components of RTI Screening Progress Monitoring School-Wide, Multi-Level Prevention System
• Primary Level• Secondary Level• Tertiary Level
Data-Based Decision Making for• Instruction • Evaluating Effectiveness• Movement Within the Multi-Level System• Disability Identification (in Accordance With State Law)
National Center on Response to Intervention
Essential Components of RTI
9
National Center on Response to Intervention 10
Screening PURPOSE: Identify students who are at risk for poor
learning outcomes FOCUS: ALL students TOOLS: Brief assessments that are valid, reliable, and
demonstrate diagnostic accuracy for predicting learning or behavioral problems
TIME FRAME: Administered more than one time per year (e.g., fall, winter, spring)
National Center on Response to Intervention 11
Progress Monitoring PURPOSE: Monitor students’ response to primary, secondary, or
tertiary instruction in order to estimate rates of improvement, identify students who are not demonstrating adequate progress, and compare the efficacy of different forms of instruction
FOCUS: Students identified through screening as at risk for poor learning outcomes
TOOLS: Brief assessments that are valid, reliable, and evidence-based
TIME FRAME: Students are assessed at regular intervals (e.g., weekly, biweekly, or monthly)
National Center on Response to Intervention 12
Data-Based Decision Making Analyze data at all levels of RTI implementation
(e.g., state, district, school, grade level) and all levels of prevention (i.e., primary, secondary, or tertiary).
Establish routines and procedures for making decisions. Set explicit decision rules for assessing student progress
(e.g., state and district benchmarks, level and/or rate). Use data to compare and contrast the adequacy of the
core curriculum and the effectiveness of different instructional and behavioral strategies.
National Center on Response to Intervention
Review Activity What is the difference between a mastery measure and a
general outcome measure? T or F: All progress monitoring tools are created equal. Where can I find evidence of the reliability and the validity of
progress monitoring tools? Name three uses of progress monitoring data. What is a trend line? What are three ways to establish progress monitoring goals? Describe two ways to analyze progress monitoring data.
13
National Center on Response to Intervention
WHAT IS A MULTI-LEVEL PREVENTION SYSTEM?
14
National Center on Response to Intervention
Essential Components of RTI
15
National Center on Response to Intervention
Levels, Tiers, and Interventions
16
FRAMEWORK3 levels of intensity:• Primary• Secondary• Tertiary
Secondary Level of
Prevention(~15% of students)
Tertiary Level of
Prevention(~5 % of
students)
Primary Level of Prevention
(~80% of students)
National Center on Response to Intervention
Levels, Tiers, and Interventions
MODELMinimum of 3 tiers representing each level of intensity
Tier II (secondary)
Tier I (primary)
17
Tier III (tertiary)
National Center on Response to Intervention
Levels, Tiers, and Interventions
Interventions are provided at each level and within each tier.
Secondary Level of Prevention
Tertiary Level of Prevention
Primary Level of Prevention
18
National Center on Response to Intervention 19
NCRTI Recommends Different Evidence Standards Across Intervention LevelsResearch-based curricula: Evidence-based intervention:
Recommended for secondary and tertiary prevention (when available)
Materials evaluated using rigorous research design
Evidence of positive effects for students who received the intervention
Recommended for primary prevention across subjects.
Components have been researched and found to be generally effective.
Curriculum materials have not been rigorously evaluated as a package.
(NCRTI, 2010)
National Center on Response to Intervention
PRIMARY PREVENTION LEVEL
20
National Center on Response to Intervention
Primary Prevention Level FOCUS: ALL students INSTRUCTION: District curriculum and instructional
practices that are research based; aligned with state or district standards; and incorporate differentiated instruction
SETTING: Regular education classroom ASSESSMENTS: Screening, continuous progress
monitoring, and outcome measures or summative assessments
21
National Center on Response to Intervention
Primary Prevention Focus ALL students Includes students with disabilities, learning
differences, or language barriers Increase access through
• Differentiated instruction• Practices that are linguistically and culturally responsive• Accommodations• Modifications
22
National Center on Response to Intervention
Primary Level Instruction Research-based curriculum materials for students
(including subgroups) Implementation fidelity Articulation of teaching and learning within and
across grades Differentiation of instruction based on data Ongoing professional development
23
See NCRTI Integrity Rubric
National Center on Response to Intervention
What Is Core Curriculum in RTI? Course of study deemed critical Usually mandatory for all students of a school or a
school system Often instituted at the elementary and secondary
school levels by local school boards, departments of education, or other administrative agencies charged with overseeing education
24
National Center on Response to Intervention
What Are Differentiated Learning Activities? Offers students in the same class different teaching
and learning strategies based on• Student assessment data and knowledge of student
readiness
• Learning preferences
• Language and culture
25
National Center on Response to Intervention
What Are Differentiated Learning Activities? Involves
• Mixed instructional groupings,• Team teaching, • Peer tutoring,• Learning centers, and • Accommodations to ensure that all students have access to
the instructional program
Is NOT the same as providing more intensive interventions to students with learning disabilities
26
National Center on Response to Intervention
Primary Prevention Setting Regular education classroom or similar setting Various grouping strategies (examples):
• Whole class • Cooperative learning groups• Peer dyads
27
National Center on Response to Intervention
Primary Prevention Assessment Universal screening to determine students’ current
level of performance Continuous progress monitoring to confirm risk status
and monitor progress of at-risk students Outcome measures or summative assessments for
accountability
28
National Center on Response to Intervention
Progress Monitoring and Screening Data Within Primary Prevention Screening data
• Identify students who need additional assessment or instruction.
• Evaluate the effectiveness of primary prevention for all students.
Progress monitoring data• Confirm and disconfirm risk.
29
National Center on Response to Intervention
Screening: Identify Students Who Need Additional Assessment and Instruction
30
10
60
50
30
40
20
70
Fall
Above average
Average
Below average
Student
Benchmark Scores for Grade 2 Screening Measure
Sco
re
National Center on Response to Intervention
Screening: Evaluate Effectiveness of Primary Prevention
31
20
120
100
60
80
40
140
Fall SpringWinter
Target score
General population
Title I
Special educationS
core
National Center on Response to Intervention
Progress Monitoring: Confirming Risk Status
32
0
5
10
15
20
25
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Weeks of Instruction
Dig
its
Co
rrec
t in
3 M
inu
tes
X
X
National Center on Response to Intervention
0
5
10
15
20
25
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Weeks of Instruction
Pro
ble
ms
Co
rrec
t in
3 M
inu
tes
XX
Progress Monitoring: Confirming Risk Status
33
National Center on Response to Intervention
Team Activity: Primary Prevention Complete column 2: Primary prevention for:
• Tier(s)• Focus• Instruction • Setting• Assessment• Data-based decision making• Other
34
See Training Manual Appendix B for
Handout
National Center on Response to Intervention
SECONDARY PREVENTION LEVEL
35
National Center on Response to Intervention
Secondary Prevention Level FOCUS: Students identified through screening as at
risk for poor learning outcomes INSTRUCTION: Targeted, supplemental instruction
delivered to small groups SETTING: Regular education classroom or other
regular education location within the school ASSESSMENTS: Progress monitoring, diagnostic
36
National Center on Response to Intervention
Secondary Prevention Focus Students identified through screening as at risk for
poor learning outcomes Typically 15%-20% of entire population
37
National Center on Response to Intervention
Secondary Level Instruction evidence-based Aligns with and supports core instruction Implementation fidelity based on developer guidelines Delivered by well-trained staff in optimal group sizes Decisions are based on valid and reliable data and
criteria are implemented accurately Supplements core instruction
38
See NCRTI Integrity Rubric
National Center on Response to Intervention
Secondary Prevention Setting Regular education classroom or similar setting Adult-led instruction Small group rather than whole class
39
National Center on Response to Intervention
Secondary Prevention Assessment Decisions about responsiveness to intervention
• Are based on reliable and valid progress monitoring data. • Reflect judgment based on the slope of improvement or
final status at the end of the intervention period. Decision-making rules are applied accurately and
consistently
40
National Center on Response to Intervention
Secondary Prevention Assessment Progress monitoring
• Monitor student response to secondary instruction.• Evaluate the efficacy of the secondary system.• Conduct at least monthly.
Diagnostic assessment• Match students’ needs to interventions.
41
National Center on Response to Intervention
Secondary Prevention Goal Setting End-of-year benchmarking National norms for weekly rate of improvement
(slope)
42
National Center on Response to Intervention
Progress Monitoring Data Within Secondary Prevention Progress monitoring data
• Determine response to secondary interventions using The four-point rule.
Trend-line analysis.• Compare efficacy of secondary interventions.
43
National Center on Response to Intervention
Progress Monitoring: Determining Response Using the Four-Point Rule
44
Goal lineX
X
National Center on Response to Intervention
Progress Monitoring: Determining Response Using Trend Line Analysis
45
X
Goal line
Trend line
X
X X
National Center on Response to Intervention
Progress Monitoring: Compare Efficacy of Secondary Interventions
0
10
20
30
40
50
Week 1 Week 2 Week 3 Week 4
Intervention A
Intervention B
Intervention C
46
Growth by Intervention Type
Wor
ds R
ead
Cor
rect
ly
National Center on Response to Intervention
Progress Monitoring: Evaluate Efficacy of Secondary System Data should indicate the following:
• Most students benefit from secondary interventions, but a small percent will need more intensive, individualized instruction (tertiary)
• Implementation fidelity for interventions and data-based decision rules
47
National Center on Response to Intervention
Team Activity: Secondary Prevention
Complete column 3: Secondary prevention for• Tier(s)• Focus• Instruction • Setting• Assessment• Data-based decision making• Other
48
See Training Manual Appendix B for
Handout
National Center on Response to Intervention
TERTIARY PREVENTION LEVEL
49
National Center on Response to Intervention
Tertiary Prevention Level FOCUS: Students who have not responded to primary
or secondary level prevention INSTRUCTION: Intensive, supplemental instruction
delivered to small groups or individually SETTING: Regular education classroom or other
appropriate setting within the school ASSESSMENTS: Progress monitoring, diagnostic
50
National Center on Response to Intervention
Tertiary Prevention Focus Students who have not responded to primary or
secondary level prevention, or who present with markedly low achievement
Typically 3%-5% of the entire population
51
National Center on Response to Intervention
Tertiary Level Instruction Evidence-based or based on validated progress monitoring
methods for individualizing instruction More intense than secondary Implementation fidelity Delivered by well-trained staff in optimal group sizes Decisions are based on valid and reliable data, and criteria are
implemented accurately. Address general education
curriculum in appropriate manner for students.
52
See NCRTI Integrity Rubric
National Center on Response to Intervention
Tertiary Prevention Setting Regular education classroom or other appropriate
setting Decisions are made on a case-by-case basis. Optimal group size is chosen for ages and needs of
students.
53
National Center on Response to Intervention
Tertiary Prevention Assessment Decisions about responsiveness to intervention
• Are based on reliable and valid progress monitoring data.• Reflect judgment based on the slope of improvement or
final status at the end of the intervention period. Decision-making rules are in place and applied
accurately.
54
National Center on Response to Intervention
Tertiary Prevention Assessment Progress monitoring
• Frequent progress monitoring (ideally weekly) is recommended.
• Continuously monitor progress based on established learning trajectories indicated by the goal line.
Diagnostic• Match instruction to needs.• Inform individualized instructional planning.
55
National Center on Response to Intervention
Tertiary Prevention: Goal Setting End-of-year benchmarking National norms for weekly rate of improvement
(slope) Intra-individual
56
National Center on Response to Intervention
Progress Monitoring Data Within Tertiary Prevention Progress monitoring data
• Determine response to secondary interventions using The four-point rule. Trend line analysis. Trend line analysis and slope.
• Compare efficacy of tertiary interventions.
57
National Center on Response to Intervention
Progress Monitoring: Determining Response Using Four-Point Rule
58
Goal lineX
X
National Center on Response to Intervention
Progress Monitoring: Determining Response Using Trend Line Analysis
59
02468
101214161820
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Weeks of Instruction
Wo
rds
Rea
d C
orr
ectl
y
X XGoal line
Trend line X
X
National Center on Response to Intervention
Progress Monitoring: Determining Response Using Trend Line Analysis and Slope
60
XStudent’s new goal and slope:
(28 – 6) ÷ 11 = 2.0 slope
02468
1012141618202224262830
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Weeks of Instruction
Wo
rds
Rea
d C
orr
ectl
y
X
Goal line
Trend line X
XX X
National Center on Response to Intervention
Progress Monitoring: Compare Efficacy of Tertiary Interventions
0
10
20
30
40
50
Week 1 Week 2 Week 3 Week 4
Intervention A
Intervention B
Intervention C
61
Growth by Intervention Type
Wo
rds
Re
ad
Co
rre
ctly
National Center on Response to Intervention
Progress Monitoring: Evaluate Efficacy of Tertiary System Data should indicate the following:
• Majority of students in tertiary prevention are demonstrating adequate progress
• Implementation fidelity for interventions, program implementation, and data decision rules
62
National Center on Response to Intervention
Team Activity: Tertiary Prevention Complete column 4: Tertiary prevention for:
• Tier(s)• Focus• Instruction • Setting• Assessment• Data-based decision making• Other
63
See Training Manual Appendix B for
Handout
National Center on Response to Intervention 64
Changing the Intensity and Nature of Instruction Intervention Duration Frequency Interventionist Group size
National Center on Response to Intervention
IDEA AND MULTI-LEVEL PREVENTION SYSTEM
65
National Center on Response to Intervention
What About Special Education?
Two groups to consider: Students with disabilities who are currently receiving
special education Students being referred for special education
eligibility consideration
66
National Center on Response to Intervention
What About Students With Disabilities?
67
~15%
~5% Tertiary
Specialized individualized systems for students with intensive needs
SecondarySupplemental group systems
for students with at-risk response to primary level
Primaryschool-wide
instruction forall Students, including
differentiated instruction
~80%
National Center on Response to Intervention
Disability Identification
To ensure that underachievement in a child suspected of having a specific learning disability is not due to a lack of appropriate instruction in reading or math, the group must consider, as part of the evaluation, what is described in 34 CFR 300.304 through 300.306:
Data demonstrate that prior to, or as a part of, the referral process, the child was provided appropriate instruction in regular education settings, delivered by qualified personnel.
Data-based documentation of repeated assessments of achievement at reasonable intervals reflect formal assessment of student progress during instruction, which was provided to the child’s parents.
68
(www.idea.ed.gov)
National Center on Response to Intervention
When Do We Refer to Special Education?
69
~15%
~5% Tertiary
Secondary
Primary
~80% of students
Example 1: After nonresponsiveness to two secondary interventions
National Center on Response to Intervention
When Do We Refer to Special Education?
70
~15%
~5% Tertiary
Secondary
Primary
~80% of students
Example 2: After nonresponsiveness to one secondary and one tertiary intervention
National Center on Response to Intervention
Recommendations Collaboration between special education and regular
education to develop an inclusive multi-level instruction model
Create written guidance on the following:• How do special education services “fit” in the model?• What are the roles and responsibilities of special
education staff?• When should students be referred for eligibility
consideration?
71
National Center on Response to Intervention
Prereferral Model Versus RTI
72
National Center on Response to Intervention
SELECTING EVIDENCE-BASED PRACTICES
73
National Center on Response to Intervention
Selecting Evidence-Based Practices: Definitions Evidence-based intervention
• Data demonstrate (or empirically validate) efficacy through scientific, rigorous research designs of the specific program
Research-based curricula• May incorporate design features that have been
researched generally• The curriculum or program as a whole has not been
studied using a rigorous research design
74
National Center on Response to Intervention
Varying Evidence Standards
Reminder: NCRTI recommends different evidence standards across intervention levels. Research-based curricula
• Recommended for primary prevention across subjects.• Components have been researched and found to be
generally effective.• Curriculum materials have not been rigorously evaluated
as a package
75
National Center on Response to Intervention
Varying Evidence Standards Evidence-based intervention
• Recommended for secondary and tertiary* prevention• Curriculum evaluated using a rigorous research design • Evidence of positive effects for students who received
the intervention
*Note: for students who have been persistently nonresponsive to interventionsthat have been administered with fidelity, strategic, data-driven individualizationof instruction may be necessary.
76
National Center on Response to Intervention
Selecting Evidence-Based Practices1. Identify needs and priorities.2. Select practices to address needs.
• Resources for identifying evidence-based practices3. Evaluate evidence claims.
• Standards of evidence across intervention levels/tiers within RTI
4. Implement practices.5. Evaluate effectiveness.
77
National Center on Response to Intervention
1) Identifying Needs and Priorities
1. Gather a team.2. Conduct a “needs assessment.”
• Gather information from multiple sources.• Compile and summarize data.
3. Determine priorities.
78
See Instructional Intervention Tools Chart
User’s Guide
National Center on Response to Intervention
Identifying Needs and Priorities Evaluate existing data to determine baseline
performance on indicators of interest.• Academic achievement• Disciplinary referrals• Attendance/truancy data
79
National Center on Response to Intervention
Identifying Needs: Interventions For what skills do we need a secondary intervention
instructional program? Is there a specific academic outcome or measure for which we are interested in providing supplemental instruction?
For what grades do we need an instructional program? Will this program be used with all students who are not
progressing in the core curriculum or only with specific sub groups of students? Primary? Secondary? Tertiary?
Which sub groups? English language learners (ELLs)? Students with disabilities?
80
See Instructional Intervention Tools Chart
User’s Guide
National Center on Response to Intervention
Identifying Priorities: Interventions Is it a program that can be purchased for a reasonable cost? Is it a program with a reasonable implementation time? Is it a program that requires specialized expertise or lengthy
training to administer? Is it a program that offers ready access to training and
technical support for staff? Is it a program that has documented evidence of efficacy
through the most rigorous research? Is it a program whose effectiveness
has been studied and demonstrated in our district or state?
81
See Instructional Intervention Tools Chart
User’s Guide
National Center on Response to Intervention
Selecting Evidence-Based Practices1. Identify needs and priorities.2. Select practices to address needs.
• Resources for identifying evidence-based practices
3. Evaluate evidence claims.• Standards of evidence across intervention levels/tiers
within RTI
4. Implement practices.5. Evaluate effectiveness.
82
National Center on Response to Intervention
2) Selecting Evidence-Based Practices Screening and progress monitoring Data-based decision making Instruction/interventions
83
See Training Manual Appendix C for more
Resources
National Center on Response to Intervention
Selecting Evidence-Based Practices: Screening and Progress Monitoring
Examples: NCRTI: Screening Tools Chart NCRTI: Progress Monitoring Tools Chart IES practice guide: RTI for mathematics and reading IRIS Center modules and RTI/assessment training
modules
84
National Center on Response to Intervention
Selecting Evidence-Based Practices: Data-Based Decision Making
Examples:
IES Practice Guide: Using Student Achievement Data
U.S. Department of Education: Doing What Works and the American Recovery and Reinvestment Act
IRIS Center Case Study Unit RTI: Data-based Decision Making
85
National Center on Response to Intervention
Selecting Evidence-Based Practices: Instruction/Intervention
Examples: What Works Clearinghouse Best Evidence Encyclopedia NCRTI Instructional Intervention Tools Chart Center on Instruction IES Practice Guide: RTI for Mathematics and Reading IRIS Center Learning Strategies Modules
86
National Center on Response to Intervention
Selecting Evidence-Based Practices1. Identify needs and priorities.2. Select practices to address needs.
• Resources for identifying evidence-based practices
3. Evaluate evidence claims.• Standards of evidence across intervention
levels/tiers within RTI
4. Implement practices.5. Evaluate effectiveness.
87
National Center on Response to Intervention
3) Evaluating Evidence Where can I find evidence? What type of evidence exists? What is the quality of the evidence? What were the desired outcomes? What are the effects of the intervention? Is the sample population similar?
88
National Center on Response to Intervention
Evaluating Evidence: Where? Curriculum websites (use with caution) Peer-reviewed journals (various)
• ERIC• Google Scholar• Education Abstracts (Education Full Text) • Psychological Abstracts
89
National Center on Response to Intervention
Evaluating Evidence: Where? What Works Clearinghouse Best Evidence Encyclopedia NCRTI Instructional Intervention Tools Chart
90
National Center on Response to Intervention
Evaluating Evidence: Type? What type of evidence is available?
• Research study• Summary of existing research• Technical report• Peer/publisher claims• Other methods?
91
National Center on Response to Intervention
Evaluating Evidence: Type? Randomized control trial* Quasi-experiment Single-case design Quantitative research synthesis
* Most rigorous form of research
92
National Center on Response to Intervention
Evaluating Evidence: Quality? How was the program implemented? Is it realistic
under normal circumstances?• Who implemented the intervention?• Was the intervention described?• Was there a manual or a script?• How often did the intervention occur?• Was fidelity evaluated in the study? If so, how?• What phase of research? (exploratory, pilot, efficacy,
scale-up)
93
National Center on Response to Intervention 94
Evaluating Evidence: Desired Outcomes? Were the outcomes assessed relevant to your
outcomes?• What outcome measures were used to evaluate the
intervention?• Do the outcome measures seem reasonable?• Are they relevant to your concerns?• Are they reliable and valid?• Is it feasible to implement with your population?
National Center on Response to Intervention
Evaluating Evidence: Effects? Are the effects large enough to be meaningful?
• Significance of differences • Interpretation of effect sizes
95
Practitioners can use this information to compare programs and identify those most likely to meet their specific needs.
National Center on Response to Intervention
Evaluating Evidence: Population? For which population does the evidence show an
effect?• Is the sample described? • Can you tell who was studied?• Is the participant similar to or representative of your
student population?• Are there different effects for different population
groups?
96
National Center on Response to Intervention
Demonstration What Works Clearinghouse Best Evidence Encyclopedia NCRTI Instructional Intervention Tools Chart
97
National Center on Response to Intervention
What Works Clearinghouse
98
Click here to select a topic
area
National Center on Response to Intervention
You can use the drop down to find topics by outcomes
You can also click on the Find what works link for a more extensive search feature
What Works Clearinghouse
99
National Center on Response to Intervention
What Works Clearinghouse
100
What outcome are you focused on?
What grade level are you looking for?
Is there a specific focus or population?
National Center on Response to Intervention
What Works Clearinghouse
101
Click here to create a report
What level of effectiveness are you looking for?
What is the extent of evidence you are looking for?
How will the intervention be delivered?
Are you looking for a curriculum, a supplemental intervention, or a practice?
National Center on Response to Intervention
What Works Clearinghouse
102
Use the filter to select your interest
Look at the results provided for the student outcome, the improvement index, the effectiveness rating, and the extent of evidence.
Click the link to learn more about the intervention and research
National Center on Response to Intervention
Evaluating Evidence
103
Click here to learn more about the interventions
National Center on Response to Intervention
Evaluating Evidence
104
National Center on Response to Intervention
Evaluating Evidence
105
Accelerated Reader™ (Reviewed for Beginning Reading)
National Center on Response to Intervention
Evaluating Evidence
106
Accelerated Reader™ (Reviewed for Beginning Reading)
National Center on Response to Intervention
Evaluating Evidence
107
Accelerated Reader™ (Reviewed for Beginning Reading)
National Center on Response to Intervention
Evaluating Evidence
108
National Center on Response to Intervention
Program Information
109
National Center on Response to Intervention
Team Activity: What Works Clearinghouse
1. What effective supplemental (Secondary or Tertiary), small group, and reading comprehension programs are available for eighth-grade students?
2. How many programs show a positive improvement index for mathematics achievement?
3. Identify a program used in your district/school• What is the improvement index, the evidence rating, and
the extent of evidence?• What additional information is available about the evidence
in the program report?
110
National Center on Response to Intervention
Best Evidence Encyclopedia
Click to select the topic:• subject matter• grade level• focus area (e.g.,
ELLs, struggling readers)
111
National Center on Response to Intervention
Finding an Evidence-Based Program
Find programs that show high or moderate levels of effectiveness
Find programs that show limited evidence of effectiveness Find programs with
insufficient evidence or no qualifying studies
See the key findings, the summary and, the methods
112
National Center on Response to Intervention
Best Evidence Encyclopedia
113
National Center on Response to Intervention
Best Evidence Encyclopedia: Review Criteria
114
Review criteria
Studies are rated based on the effect size, the type of study, and the size of study
National Center on Response to Intervention
Selecting an Evidence-Based Program
115
National Center on Response to Intervention
Evaluating Evidence
116
National Center on Response to Intervention
Determine the Type of Program
117
The type of program is defined as
National Center on Response to Intervention
Learn About the Program
118
The program name, a brief description, and contact information for more information are provided.
National Center on Response to Intervention
Team Activity: Best Evidence Encyclopedia
1. What programs show strong or moderate levels of effectiveness for eighth-grade reading?
2. Were any small group tutorials (SGT in Column 3) shown to have a strong level of effectiveness for struggling readers?
3. Identify a program used in your district/school:• What level of evidence is reported for the program?
119
National Center on Response to Intervention
NCRTI Instructional Intervention Tools Chart
120www.rti4success.org
National Center on Response to Intervention
NCRTI Instructional Intervention Tools Chart The tools chart lists instructional programs that can
be used as secondary interventions within an RTI context.
The technical review committee (TRC) does not rate instructional programs; instead the TRC rates studies of program efficacy.
121
National Center on Response to Intervention
NCRTI Instructional Intervention Tools Chart
NCRTI definition of instruction: Additional or alternative intervention programs to the core
curriculum conducted in small groups or individually with evidence of efficacy for improving academic outcomes for students whose performance is unsatisfactory in the core program.
122
National Center on Response to Intervention
NCRTI Instructional Intervention Tools Chart Purpose: help consumers identify secondary level
intervention programs that• Have been evaluated through rigorous design.• Have shown positive, meaningful treatment effects.
123
National Center on Response to Intervention 124
Search by Content or Grade Level
Search by Content: Reading, Math, or Writing
Search by Grade: Elementary or Secondary
National Center on Response to Intervention 125
Comparing Interventions or Studies
Narrow search selection to interventions that fit your needs
National Center on Response to Intervention
Comparing Interventions or Studies
126
Compare them side by side
National Center on Response to Intervention 127
NCRTI Instructional Intervention Tools Chart
Click on the name of the program to see the implementation requirements.
National Center on Response to Intervention 128
Implementation Requirements
National Center on Response to Intervention
Technical Rigor of the Study
129
Study Quality
National Center on Response to Intervention
Participants
130
National Center on Response to Intervention
Design
131
National Center on Response to Intervention
Fidelity of Implementation
132
National Center on Response to Intervention
Measures
133
National Center on Response to Intervention
Technical Rigor of the Study
134
Effect Size
Adjusted posttest: corrects for differences in groups on pretest
National Center on Response to Intervention
Technical Rigor of the Study
135
Effect Size
Unadjusted posttest: does not account for differences in groups on pretest
National Center on Response to Intervention
Effect Size: Sample 1
136
Developer was unable to provide necessary data for NCRTI to calculate effect sizes.
National Center on Response to Intervention
Effect Size: Sample 2
137
Mean for proximal and distal measures
National Center on Response to Intervention
Effect Sizes
138
National Center on Response to Intervention
Disaggregated Data The column reports effect size data that have been
disaggregated for sub groups, if available.• Students with disabilities• ELLs• Students from diverse racial-ethnic groups
139
National Center on Response to Intervention
Team Activity: Instructional Intervention Tools Chart
1. What secondary instruction programs are available for elementary mathematics students?
2. Which study has the highest average adjusted effect size?
3. What are the average unadjusted effect sizes for SSRD Writing Strategies?
4. Identify a program used in your district/school• What level of evidence is reported for the program?
140
National Center on Response to Intervention
Example 1: Bell Top Elementary SchoolMs. Jones, a third-grade math teacher, has noticed that Jack, Bobby, and Jane seem to be struggling with word problems despite the strong research-based core curriculum. In looking at their scores on the screening measure, Ms. Jones noticed that they are all below the cut score. After monitoring their progress over a number of weeks she continued to see no progress. Can you identify an intervention that might help Jack, Bobby, and Jane improve? Why did you select that intervention?
141
See Training Manual Appendix D
National Center on Response to Intervention
Example 2: Lake Ridge Middle SchoolMr. Morris has identified that Jessica does not have a strong grasp on the basic skills of reading and is therefore struggling in his class. After looking at her screening data and monitoring her progress, he has decided to implement an intervention to help her with basic reading skills. Is there an intervention available that Mr. Morris might use to help Jessica? What questions might he ask in selecting an appropriate intervention?
142
See Training Manual Appendix D
National Center on Response to Intervention
Team Activity: Selecting Evidence-Based Practices Choose a content area Think of intervention materials you have or are
considering adopting Using the three sites, investigate the evidence for
that program
143
National Center on Response to Intervention
Selecting Evidence-Based Practices1. Identify needs and priorities.2. Select practices to address needs.
• Resources for identifying evidence-based practices3. Evaluate evidence claims.
• Standards of evidence across intervention levels/tiers within RTI
4. Implement practices.5. Evaluate effectiveness.
144
National Center on Response to Intervention
4) Implement Practices Provide initial recommended training and
professional development. Plan for initial implementation (e.g., scheduling,
materials). Provide ongoing coaching and professional
development. Monitor and evaluate fidelity of implementation.
145
National Center on Response to Intervention
What Is Fidelity of Implementation?
146
National Center on Response to Intervention
Implement Practices: Fidelity
The best way to monitor fidelity is to measure it. Self-Report Data Observation Logs, lesson plans, and student work
147
National Center on Response to Intervention 148
Evaluating Implementation: Monitoring Fidelity
Self-Report Data Questionnaires, surveys, interviews May provide an indicator of teacher knowledge,
context of implementation Often unreliable when used alone
National Center on Response to Intervention 149
Evaluating Implementation: Monitoring Fidelity
Evaluating fidelity through observation Develop checklists of critical implementation components Record and listen to sessions at random Spot checks Peer observations Peer coaching
National Center on Response to Intervention 150
Evaluating Implementation: Monitoring Fidelity
Logs, lesson plans, and student work Allows for evaluation of what was done
• Content covered• Student progress
Less information about:• Delivery• Dosage• Adherence to scripts (if applicable)
National Center on Response to Intervention 151
Evaluating Implementation
Using Fidelity Data
Distinguish curriculum/intervention from quality of implementation when problems occur
National Center on Response to Intervention 152
Evaluating Implementation
Using Fidelity Data
Identify implementation strengths • People • Potential coaches • Components of intervention
Target areas in need of improvement• Coaching• Professional development• Retraining
National Center on Response to Intervention
Selecting Evidence-based Practices
1. Identify needs and priorities.2. Select practices to address needs.
• Resources for identifying evidence-based practices
3. Evaluate evidence claims.• Standards of evidence across intervention levels/tiers
within RTI
4. Implement practices.5. Evaluate effectiveness.
153
National Center on Response to Intervention
5) Evaluate Effectiveness Evaluate general intervention effectiveness. Evaluate effectiveness for individual students.
• Ongoing, graphed progress monitoring data• Reviewed at least every 4-6 weeks• Evidence-based decision-making criteria• General outcome measure versus mastery measure
154
National Center on Response to Intervention
Compare Efficacy of Interventions
0
10
20
30
40
50
Week 1 Week 2 Week 3 Week 4
Intervention A
Intervention B
Intervention C
155
Growth by Intervention Type
Wor
ds R
ead
Cor
rect
ly
National Center on Response to Intervention
Intervention Effectiveness
156
Intervention A
Goal line
Trend line
Trend lineIntervention B
Indicates Change
National Center on Response to Intervention
CLOSING
157
National Center on Response to Intervention
Things to Remember Good data IN … Good data OUT
• Know where your data came from and the validity of that data
Focus on the big picture or ALL students• Are most students making progress?
ALL instructional and curriculum decisions should be based on DATA.
Keep it SIMPLE and EFFICIENT!
158
National Center on Response to Intervention
Implementing the RTI Framework Select and implement evidence-based practices and
procedures. Implement essential components with integrity. Ensure tools and implementation reflect the cultural,
linguistic, and socioeconomic factors that students bring to the classroom.
159
National Center on Response to Intervention
Review Activity Why does the NCRTI use levels instead of tiers? Where can districts and schools find evidence of an
intervention’s effectiveness? How can schools monitor the fidelity of intervention
implementation?
160
National Center on Response to Intervention
Next Steps
Consider developing a district/school multi-level prevention framework model (e.g., guidance document): http://www.rti4success.org/resourcetype/ information brief-developing-rti-guidance-document
Develop an RTI district implementation plan.
161
National Center on Response to Intervention 162
National Center on Response to Interventionwww.rti4success.org
RTI Action Networkwww.rtinetwork.org
IDEA Partnershipwww.ideapartnership.org
Need More Information?
National Center on Response to Intervention
National Center on Response to Intervention
www.rti4success.org
Questions?
163
National Center on Response to Intervention 164
National Center on Response to InterventionThis document was produced under U.S. Department of Education, Office of Special Education Programs Grant No. H326E070004 Grace Zamora Durán and Tina Diamond served as the OSEP project officers. The views expressed herein do not necessarily represent the positions or policies of the Department of Education. No official endorsement by the U.S. Department of Education of any product, commodity, service or enterprise mentioned in this publication is intended or should be inferred. This product is public domain. Authorization to reproduce it in whole or in part is granted. While permission to reprint this publication is not necessary, the citation should be: www.rti4success.org.
top related