florida assessments for instruction in reading (fair) : an overview and application within a...

Florida Assessments for Instruction in Reading (FAIR) : An Overview and

Application within a Response to Intervention (RTI) Model

Liz Crawford, MS, CCC-SLPDirector of Interventions

November 12, 2009IDA – Orlando, FL

Agenda Overview of FAIR

K-2 3-12

Progress Monitoring & Reporting Network (PMRN) Reports

Common Profiles/Patterns in Data Connecting to Instruction Response to Instruction/Intervention (RTI) Model RTI and Special Education Identification RTI in Florida RTI and FAIR

Why New Reading Assessments?

Primary-grade assessment improved to include: 1. PreK; 2. Vocabulary; 3. Comprehension

Assesses standards in K-12; monitors progress Results predict end of year performance on standardized test

(K-2) & Florida’s Comprehensive Assessment Test (FCAT) Diagnostic inventories are linked to Florida standards and

provide information for guiding instruction Reading comprehension & oral reading fluency passages are

equated for difficulty to allow for accurate progress monitoring and comparison of student performance from one passage to another

Comprehension assessed in instructional level passages

K-2: phonemic awareness, letter knowledge, decoding, encoding, fluency, vocabulary, listening or reading comprehension

3-12: adaptive, complex & low level reading comprehension, fluency, word analysis, skill assessment

Florida Assessments for Instruction in Reading:(Teacher Administered)

Comprehensive Assessment System

1. Phonemic Awareness

2. Phonics

3. Fluency

4. Vocabulary

5. Text Comprehension

6. Orthographic Skills (Spelling)

K-2 3-12

*

* embedded in text comprehension

K-2 Assessments

The K-2 “Big Picture” Map

Broad Screen/Progress Monitoring Tool (BS/PMT) “All” students

• Letter Naming & Sounds• Phonemic Awareness • Word Reading

Broad Diagnostic Inventory (BDI) “All” students“Some” students for vocabulary

• Listening Comprehension• Reading Comprehension• Vocabulary • Spelling (2nd grade only)

Targeted Diagnostic Inventory(TDI) “Some” students

• K = 9 tasks• 1st = 8 tasks• 2nd = 6 tasks

Ongoing Progress Monitoring(OPM)“Some” students

•K – 2 = TDI tasks•1 – 2 = ORF

7

The K-2 Broad Screen/Progress Monitoring Map

K-2 Broad Diagnostic Inventory (BDI) Map

Kindergarten • AP 1 = Listening Comprehension (LC) and Vocabulary (VOC)• AP 2 = Listening Comprehension• AP 3 = Listening Comprehension or Reading Comprehension (RC) and Vocabulary

First Grade • AP 1 = Reading Comprehension or Listening Comprehension and Vocabulary• AP 2 = Reading Comprehension• AP 3 = Reading Comprehension and Vocabulary

Second Grade • AP 1 = Reading Comprehension, Vocabulary, and Spelling (SPL)• AP 2 = Reading Comprehension and Spelling• AP 3 = Reading Comprehension, Vocabulary, and Spelling

K-2 Targeted Diagnostic Inventory (TDI) MapKindergarten • Print Awareness

• Letter name and sound knowledge• Phoneme Blending• Phoneme Deletion Word Parts/Initial• Letter Sound Connection Initial• Letter Sound Connection Final• Word Building –Initial Consonants• Word Building –Final Consonants• Word Building –Medial Vowels

First Grade • Letter Sound Knowledge • Phoneme Blending• Phoneme Deletion Initial• Phoneme Deletion Final• Word Building –Consonants• Word Building –Vowels• Word Building –CVC /CVCe• Word Building –Blends

Second Grade • Phoneme Deletion Initial• Phoneme Deletion Final• Word Building –Consonants• Word Building –CVC /CVCe •Word Building –Blends & Vowels• Multisyllabic Word Reading

Targeted Diagnostic Task Example Word Building Tasks are given at each grade

level.

Grade 1 Vowels. Student moves letters to makenew words like dig, dug, pug, pig, and pit.

Practice and Task Items

u p t a e i

dog

The K – 2 “Score” MapBS/PMT PRS = Probability of Reading Success

BDI LC = Listening Comprehension Total questions correct (implicit/explicit)

RC = Reading Comprehension Total questions correct (implicit/explicit), Fluency, Percent Accuracy Target Passage

VOC = Vocabulary Percentile Rank

SPL = Spelling Percentile Rank

TDI ME = Meets Expectations

BE = Below Expectations

OPM ORF = Adjusted Fluency

OPM TDI Tasks = ME or BE and Raw Score 12

Probability Score – Probability of Reading Success (PRS) Score calculated based on the BS/PMT

Probability is…the relative possibility that an event will occur or is likely to occur.

A .70 means that we expect this student to have a 70% chance of scoring at or above the 40th percentile at the end of the year on the SESAT or SAT-10.

13

Success Zones - Probability Scores

GREEN 85% or better chance of scoring at or above the 40th percentile at the end of the year on the SESAT or SAT-10

16-84% chance of scoring at or above the 40th percentile at the end of the year on the SESAT or SAT-10

15% or less chance of scoring at or above the 40th percentile at the end of the year on the SESAT or SAT-10

14

YELLOW

RED

3-12 Assessments

Why use a computer adaptive test? What are the benefits?

A computer adaptive test (CAT) provides a more reliable assessment of student ability in a shorter amount of time than a traditional test, because it picks questions at a level of difficulty that are most informative for each student.

Computer adaptive tests provide more reliable assessments particularly for students at the extremes of ability (extremely low ability or extremely high ability).

17

The 3-12 “Big Picture” Map

Type of Assessment Name of Assessment

Broad Screen/Progress Monitoring Tool (BS/PMT) – Appropriate for ‘All’ students

• Reading Comprehension (RC)

Targeted Diagnostic Inventory(TDI) – “Some” students

• Maze• Word Analysis (WA)

Ongoing Progress Monitoring (OPM) – “Some” students

• Maze• ORF• RC

Informal Diagnostic Toolkit(Toolkit) – “Some” students

• Phonics Inventory• Academic Word Inventory• Lexile Passages• Scaffolded Discussion Templates

18

The 3-12 “Big Picture” Map

Type of Assessment Name of Assessment

Broad Screen/Progress Monitoring Tool (BS/PMT)

• 3 times a year• On the computer• Entered into PMRN

Targeted Diagnostic Inventory(TDI)

• 3 times a year • On the computer• Entered into PMRN

Ongoing Progress Monitoring (OPM)

• Maze and RC – on computer, up to 7 times a year, entered into PMRN• ORF – paper and pencil, up to 7 times a year, entered into PMRN

Informal Diagnostic Toolkit(Toolkit)

• Paper and Pencil only• Not entered into PMRN• No restrictions on when to administer

Broad Screen/Progress Monitoring Tool

Reading Comprehension Task

(3 Times a Year)

Targeted Diagnostic Inventory

Maze & Word Analysis TasksDiagnostic

Toolkit(As Needed)

OngoingProgress

Monitoring(As Needed)

If necessary

Grades 3-12 Assessments Model

Purpose of Each 3-12 Assessment RC Screen

Helps us identify students who may not be able to meet the grade level literacy standards at the end of the year as assessed by the FCAT without additional targeted literacy instruction. 

Mazes Helps us determine whether a student has more

fundamental problems in the area of text reading efficiency and low level reading comprehension. 

Word Analysis Helps us learn more about a student's fundamental

literacy skills--particularly those required to decode unfamiliar words and read accurately. 

3-12 Targeted Diagnostic Inventory (TDI) Map

Moderate or Low Success

Probability on FCAT=

Administer TDI

Word Analysis TDI

Intervention withword recognition skills?

Intervention with reasoning skills, prior knowledge,

and/or vocabulary?

Maze TDI

Interventionwith fluency/

comprehension skills?

The Broad Screen/Progress Monitoring Tool: The RC Task 10-30 minute computer-adaptive task of reading

comprehension A student receives a minimum of one passage and a

maximum of three passages Predicts student’s FCAT success probability (FSP) Administered to all students scoring Level 1 or 2 on FCAT Administered to 3rd grade students identified with a reading

deficiency by their districts Other students may take the assessment at the discretion of

their districts

TDI: The Maze Task Two 3-minute (grade-level) passages

Administered 3 times a year

Student responds to embedded cloze items within text

Assesses text reading efficiency (i.e., reading accuracy and speed, and gist - level comprehension)

TDI: The Word Analysis Task

A 5-15 minute computer-adaptive spelling task that assesses a student’s understanding of letter/sound correspondence

A student receives a minimum of 5 words and a maximum of 30 words

Results indicate whether probability of success in reading is hampered by difficulties with word-level skills.

33

The 3-12 “Score” Map

Reading Comprehension - BS/PMT

FCAT Success Probability (FSP) Color- coded

Percentile Standard Score Lexile Ability Score and Ability Range FCAT Cluster Area Scores

Maze - TDI Percentile Standard Score Adjusted Maze Score

Word Analysis - TDI Percentile Standard Score Ability Score (WAAS)

OPM RC – Ability Score, Ability Range, Cluster Scores Maze – Adjusted Maze Score ORF (3rd – 5th) Adjusted Fluency Score

Ongoing Progress Monitoring (OPM)

Ongoing Progress Monitoring (OPM)K-2 All of the TDI tasks are available as OPM probes ORF in grades 1 and 23-12 ORF at grades 3-5 MAZE at grades 3-12 RC at grades 3-12 Informal toolkit with:

Scaffolded Discussion Templates Lexile® Passages Phonics Inventory Academic Word Inventory

The purpose of frequent progress monitoring

Progress monitoring during the year is used to determine whether or not students are responding adequately to their current instructional environment.

If their growth within their current environment is determined to be inadequate, they should experience an instructional change designed to increase their rate of progress.

There are different types and purposes for progress monitoring, which call for a range in the frequency of administration (e.g., weekly, monthly).

Issues involved in deciding on the frequency of progress monitoring

Setting a reasonable interval for progress monitoring actually depends on three pieces of information about the reading assessment being used:

1. How much growth on a given outcome (i.e., oral reading fluency) can be reasonably expected during a given interval of time?

2. How reliably does the assessment measure the outcome of interest?

3. How closely are the different probes of the ability equated for difficulty?

General issues about the frequency and utility of different kinds of progress monitoring assessments

Measures like oral reading fluency are referred to as “general outcome” curriculum based measures because they assess student proficiency on a general outcome of interest.

They have the advantage of being able to determine growth rates for individual students that can be compared against a desirable standard of growth over time.

Thus, they can indicate whether or not an intervention is sufficiently powerful to “close the gap” on the outcome measure over a reasonable period of time.

Although they provide information about the necessity for a change in a student’s instructional environment, they do not necessarily provide information about how the program should be changed in order to increase student performance.

General issues about the frequency and utility of different kinds of progress monitoring assessments

Information that can be used to suggest ways that instruction should be changed is more likely to be provided by assessments that are very closely tied to the specific curriculum that is being used to guide instruction.

This type of assessment, frequently referred to as “classroom-based formative assessment”, should be given very frequently to assess whether the student is learning what is being taught.

Although this type of assessment has immediate value for making day-to-day adjustments to instruction, it has less value as a measure of general progress in learning to read, since it represents such a small part of the overall reading process.

Examples of “general outcome” curriculum based measures Oral Reading Fluency (ORF)

OPM Maze Task

OPM Reading Comprehension

OPM TDI Tasks – OPM*

*These tasks form a developmental continuum where 80% mastery is expected. Thus, these are similar to what many teachers might use for classroom-based formative assessment.

Examples of “classroom-based formative assessment” Unit Tests

Weekly Quizzes

Writing Samples

Informal Inventories

In summary… General outcome measures should not be given more

frequently than is necessary to establish a reliable estimate of growth.

Giving them more frequently than that takes time away from instruction unnecessarily.

In contrast, teachers should be constantly (hourly, daily, weekly) acquiring formative assessment data about their student’s progress in mastering their specific lesson objectives.

References for OPM Jenkins, J. R., Graff, J. J., & Miglioretti, D. L. (2009).

Estimating reading growth using intermittent CBM progress monitoring. Exceptional Children, 75, 151-163.

Use of ongoing progress monitoring to improve reading instruction. (2009). A guidance document prepared by the Florida Center for Reading Research (FCRR).

Torgesen, J. K., & Miller, D. H. (2009). Assessments to guide adolescent literacy instruction. Portsmouth, NH: RMC Research Corporation, Center on Instruction.

PMRN Reports

K-2 and 3-12

What is an AHM?The Active Header Menu (AHM) is found at the top of most reports. It allows the User to navigate between reports and filter the data that is displayed.

In the report below, it is AP3 and the teacher is filtering her second grade class so that the report only displays those students who have been in her class all year. It also provides links to legends that help explain the report and a link for “printer friendly” version of the report.

School Status Report (K-2)

Target Passages for Grades 1 and 2 – BDI - Reading Comprehension

©2009 Florida Center for Reading Research

School Status Report (3-12)

School Progress Report (K-2 example)

Bar Graph View of Data

School Progress Report (K-2 example)

Tabular View of Data

School Grade Summary Report (K-2)

School Grade Summary Report (K-2), - continued

School Grade Summary Report (K-2)

School Grade Summary Report (3-12)

School Grade Summary Report (3-12)- continued

School Grade Summary Report (3-12)

School Demographic Report (K-2 example)

School PRS Distribution Graph by Ethnicity (K-2 example)

District and School Data File The data file allows all of the data to be downloaded so that Users can

complete their own analyses. If the PMRN does not have a report comparing the groups of interest, then the

data file allows a school or district to download the data for analysis using other systems such as Excel or SPSS.

The file can be downloaded at the district or school level. The file is a text file.

Each school has its own file and it can be downloaded by the SL1, SL2, SL3, and SL4 Users at the school (up to 7 people at the school have access).

The process for accessing the file is described at http://www.fcrr.org/pmrn/download2access_excel.htm.

Student Ongoing Progress Monitoring (OPM) ORF Report

Grades 1-5

©2009 Florida Center for Reading Research

62

Instructional Changes-Pop up

Class Status Reports and Student Score Detail Boxes

K – 2 and 3 – 12

Class Status Report (K-2)-

©2009 Florida Center for Reading Research

Student Score Detail Box (K-2)-

©2009 Florida Center for Reading Research

Class Status Report- (3-12)

Student Score Detail Box-

(3-12)

Some Common Profiles or Patterns in Data

K-2 Class Status Report Worksheet – data analysis questions

How is comprehension impacted?

Assessment/Curriculum Decision Tree for Reading Improvement for the Florida Assessments for Instruction in Reading

72

Common Profiles/Patterns of Performance

73

Common Profiles/Patterns of Performance

Box 1

Box 2 + 4

Box 2 + 5

Box 3 + 4

Box 3 + 5

74

Steps to the Activity

Sort your students according to the profiles presented and write their names down on the worksheet under the appropriate profile.

After your students are sorted into profiles, answer the following questions: What profiles most frequently occurred in your student

data? For which profiles do you have resources currently in

place? In which areas do you need more professional development/support/resources to meet student needs?

Based upon the frequency of various profiles of student needs, will you address these through adjustment of Tier I, Tier II, or Tier III?

75

Assessment/Curriculum Decision Tree for Reading Improvement Grades 3-12 for the Florida Assessments for Instruction in Reading Worksheet - Handout 8

©2009 Florida Department of Education

Connecting Assessment to Instruction

Links to Instructional Resources

77

Linking to Resources through PMRN

Teachers will have the option of clicking on column headers for: K-2

RC, LC, Vocabulary, Spelling, and TDI Tasks 3-12

RC, Maze, WA These links will take them to a search page which will in turn bring

them to a list of resources designed to assist instruction in the chosen area.

Resources include K-2: Empowering Teacher Routines and Student Center

Activities 3-5: Empowering Teacher Routines and Student Center

Activities K-12: LEaRN videos and other LEaRN resources

You can also get to some of these same resources through a search tool located on FCRR home page (www.fcrr.org)

Search Tool for FCRR resources

For Student Center Activities (SCAs) for K-5 and for Empowering Teachers (ET) Routines K-3

There are two search tools: one links to FAIR and one links to DIBELS Searchable by grade level and then

5 components and then skills SSS and then enter standard number DIBELS subtest or FAIR The skills on the class status report worksheet match up

with the skills on the search tools under each component of reading instruction!

78

Parent Letters

You may access the Parent Letters in the PMRN in two different ways: From the Class Status Report click on the link at

the bottom of the page that says Print all Parent Letters – this will print letters for the whole class

Users can get to the Parent Letters under the Student Reports tab – this will allow the teacher to print only letters for specific students.

79

Response to Instruction/Intervention

What is RtI? (NRCLD)National Research Center on Learning Disabilities

“ RTI is an assessment and intervention process for systematically monitoring student progress and making decisions about the need for instructional modifications or increasingly intensified services using progress monitoring data.” pg. i.2

What is RtI? (NASDSE)National Association of State Directors of Special Education

RtI is the practice of (1) providing high-quality

instruction/intervention matched to student needs (2) using learning rate over time and level of

performance to (3) make important educational decisions.

Essential Components Implementation (NASDSE)

Multi-tier model of instruction Problem solving method An integrated data collection/assessment

system

Components 1 & 2: Tiered Model of School Supports & the Problem-Solving Process

ACADEMIC and BEHAVIOR SYSTEMS

Tier 3: Intensive, Individualized, Interventions.

Individual or small group intervention.

Tier 2: Targeted, Supplemental Interventions & Supports.

More targeted interventions and supplemental support in addition to the core curriculum and school-wide

positive behavior program.

Tier 1: Core, Universal Instruction & Supports.

General instruction and support provided to all students in all

settings.

Essential Component 3:Integrated Assessment Systems (NASDSE)

• Directly assess specific skills in standards• Assess “marker variables” [demonstrated to lead to

the ultimate instructional target, (e.g., reading comprehension)]

• Sensitive to small amounts of growth• Brief• Repeatable• Easy to use• Direct relationship to instructional decision- making

RtI and Special Education Identification

Sec. 614 (b) (6)

Specific Learning Disabilities (A) IN GENERAL

“…a local educational agency shall not be required to take into consideration whether a child has a severe discrepancy…”

Sec. 614 (6) Specific Learning Disabilities

(B) ADDITIONAL AUTHORITY

“…a local educational agency may use a process that determines if the child responds to scientific, research-based intervention as a part of the evaluation procedure…”

Process refers to “Problem Solving Process” Responds refers to “Response to Intervention”

Problems with the traditional system “Wait-to-fail model” (Vaughn and Fuchs 2003; NASDSE)

Late identification for students in need Imprecise screening through teacher observation False negatives (unidentified students) who don’t

get the services they need Identification measures that are not linked to

instruction Separation of general and special education Over representation of some minority groups Undocumented benefits

Special Education Eligibility Comparison of historical system to RtI system

See NASDSE documents See Vaughn and Fuchs article, 2003

It is not about getting them in, it is about getting them the help they need.(“If I could just get my child into special education everything would be fine.” – a parent)

Specific Learning Disabilities (SLD) Rule in Florida

Rule Implementation Synopsis Status

Adopted by SBE on January 21, 2009 Effective as of March 23, 2009

Implementation SP&P identifies applicable process school-by-school

A. RtI Process within Comprehensive Evaluation OR B. RtI Process and relevant pattern of strengths and

weaknesses within Comprehensive Evaluation Effective July 1, 2010, schools can no longer require

pattern of strengths and weaknesses as allowed in option B

96

Florida SLD Criteria for Eligibility(until July 1, 2010)

Underachievement in:Oral expression

Listening comprehensionWritten expressionBasic reading skills

Reading fluency skillsReading comprehensionMathematics Calculation

Mathematics problem-solving

RtI & Pattern of Strengths/Weaknesses:

RtI & pattern of strengths and

weaknesses relevant to identifying SLD

Conditions 1 and 2 not primarily the result of:

Visual, hearing or motor disability

Intellectual disabilityEmotional/Behavioral

disabilityCultural factors

Irregular attendanceEnvironmental or

economic disadvantageClassroom behavior

Limited English proficiency

Condition 1 Condition 2 Condition 3

+ +

RtI:Insufficient response to

scientific, research-based intervention

or

Florida SLD Criteria for Eligibility(after July 10, 2010)

Underachievement in:Oral expression

Listening comprehensionWritten expressionBasic reading skills

Reading fluency skillsReading comprehensionMathematics Calculation

Mathematics problem-solving

RTI:Insufficient response to

scientific, research-based intervention

Conditions 1 and 2 not primarily the result of:

Visual, hearing or motor disability

Intellectual disabilityEmotional/Behavioral

disabilityCultural factors

Irregular attendanceEnvironmental or

economic disadvantageClassroom behavior

Limited English proficiency

Condition 1 Condition 2 Condition 3

+ +

Collection of Current Technical Assistance

99

NEW SLD Web link on the BEESS Web site featuring related rules, technical assistance,

documents, and Web sites:

http://www.fldoe.org/ese/sldr.asp

What does RtI look like in Florida?

www.florida-rti.org

FAIR and RTI

109

RtI and K-2 Florida Assessments for Instruction in Reading (FAIR)

Broad Screen/Progress Monitoring Tool

(BS/PMT)

“All” students

Tier 1

Broad Diagnostic Inventory

(BDI)

“All” students

“Some” students for vocabulary

Tier 1 and 2 Sometimes Tier 3

Targeted Diagnostic Inventory

(TDI)

“Some” students

Tier 2 and 3

Ongoing Progress Monitoring

(OPM)

“Some” students

Tier 2 and 3

109

Guiding Questions for Determining Target Skill(s) How well is core instruction meeting the

needs of our students in the areas of: Phonemic awareness? Phonics/Word Analysis? Word Reading? Reading connected text accurately? Vocabulary? Comprehension?

School Status Report (K-2)

112

RtI and 3-12 Florida Assessments for Instruction in Reading (FAIR)

Broad Screen/Progress Monitoring Tool (BS/PMT) – Appropriate for ‘All’ students

Tier 1

Targeted Diagnostic Inventory

(TDI) – “Some” students

Tier 2 and 3

Ongoing Progress Monitoring

(OPM) - “Some” students

Tier 1 – RC OPM Tier 2 and 3

Maze and ORF OPM

Informal Diagnostic/Progress Assessment Toolkit

“Some” students

Tier 2 and 3

School Status Report (3-12)

School Grade Summary Report (3-12)

School Grade Summary Report (3-12)- continued

116

Common Profiles/Patterns of Performance

Questions??