Interpreting Literacy Data for Effective Instructional Grouping

Grades 2-5

Moira K. McKennaNorth West Positive Behavior Intervention and

Support Network Conference March 8, 2010

McKenna (2010)

Academic Systems Behavioral Systems

1-5% 1-5%

5-10% 5-10%

80-90% 80-90%

Intensive, Individual Interventions•Individual Students•Assessment-based•High Intensity

Intensive, Individual Interventions•Individual Students•Assessment-based•Intense, durable procedures

Targeted Group Interventions•Some students (at-risk)•High efficiency•Rapid response

Targeted Group Interventions•Some students (at-risk)•High efficiency•Rapid response

Universal Interventions•All students•Preventive, proactive

Universal Interventions•All settings, all students•Preventive, proactive

Designing School-Wide Systems for Student Success

McKenna (2010)

1-5% 1-5%

5-10% 5-10%

80-90% 80-90%

Intensive, Individual Interventions•Small Group/Individual students •Assessment-based•High Intensity

Intensive, Individual Interventions•Individual students•Assessment-based•Intense, durable procedures

Targeted Group Interventions•Some students (some risk)•High efficiency•Rapid response

Targeted Group Interventions•Some students (at-risk)•High efficiency•Rapid response

Universal Interventions•All students•Preventive, proactive

Universal Interventions•All settings, all students•Preventive, proactive

EBISSEBISSEffective Behavioral and Instructional Support Systems

Academic Systems Behavioral Systems

McKenna (2010)

Curriculum-Based Measurement (CBM) Definition and Evidence-Base

• CBM is a brief, standardized assessment that documents student achievement through a systematic sampling of skills that represent the annual curriculum (Fuchs, 2004; Shinn, 2002, 1998, 1989; Deno, 1986)

• Alternate passages are of equivalent difficulty, whereby each measure is represented by the same level of complexity, gaining an accurate measure of student growth

• Growth is measured by Universal Screening and Progress Monitoring

McKenna (2010)

Types of CBM • General Outcome Measures (GOM)

– application of skill to independent task – leveled passages that can be used for progress

monitoring• Skills-Based Measures (SBM)

– leveled measures that assess proficiency on a specific set of skills that students are expected to perform per grade-level standards

– Most commonly seen in mathematics/mixed math computation

• Mastery Measures (MM)– Focuses on student attainment of finite skills– not appropriate for progress monitoringMcKenna (2010)

Utility of CBMs• Screening Decisions

– identify which students may need instructional support

• Progress Monitoring Decisions– decide when to modify instruction, teach new

skills, and/or revise goals• Diagnostic Decisions

– to target specific skill(s) for support• Outcome Decisions

– to modify instruction, change intervention, or reintegrate back into general education support

McKenna (2010)

CBM as Convergent Data

• Technically reliable and valid GOMs and SBMs will be used for Universal Screening and Progress Monitoring of student performance

• MM will be used to determine if a student is able to present skills taught in a lesson or unit

• Student performance measures from these, and other relevant sources of information, will be used to determine student growth as aligned with standards

McKenna (2010)

Big idea General Outcome Measure

1. Phonological Awareness

2. Alphabetic Principal

3. Fluency (& accuracy) with connected text

4. Vocabulary

5. Comprehension

Initial Sound Fluency

Nonsense Word Fluency

DIBELS Oral Reading Fluency (DORF)

DIBELS Oral Reading Fluency (DORF); AIMSweb MAZE

Phoneme Segmentation

•DIBELS: Dynamic Indicators of Basic Early Literacy Skills•DIBELS is an example of a measurement system• AIMSweb: Letter Sound Fluency (LSF; Alphabetic Principle ) • (Fuchs & Fuchs, 2004): Word Identification Fluency (WIF; Alphabetic Principle) McKenna (2010)

Early Literacy• Skills of Phonemic Awareness assessed in fall,

winter and spring– Ability to manipulate sounds auditorily at the

phoneme level– foundation skill set required to become a proficient

reader – phoneme segmentation as capstone skill

representing knowledge in rhyme, on-set rime, and blending

– general outcome measures include Initial Sound Fluency (ISF), and Phoneme Segmentation Fluency (PSF)

McKenna (2010)

Early Literacy• Letter naming

– Adds confidence that a student is on track to be a successful reader

– Does not link to one of the five essential components of beginning reading

• Phonics / Alphabetic Principle– the ability to link letters to their representative

sounds in text– general outcome measures (GOMs) include letter-

sound fluency (LSF) and nonsense word fluency (NWF).

McKenna (2010)

Early Literacy Beginning Reading• Automaticity with Code

– Reading accurately and fluently with prosody – Measured by rates of Oral Reading Fluency (ORF)– “Best” versus “fastest” reading

• Vocabulary– Gain meaning from words and word parts– Technically adequate general outcome measure not available;

unstandardized measure through error analysis of ORF – Measured by in-program assessment

• Comprehension– Gaining meaning from text– Measured efficiently by MAZE task

McKenna (2010)

Oral Reading Fluency• Mid-First grade through 6th grade

– Norms available through 8th, 12th grade using 8th grade passages

• DIBELS and AIMSweb: 2 examples of measurement systems – http://www.rti4success.org/

• Students who’ve met benchmark, maintain instructional program– differentiate through small group instruction

McKenna (2010)

MAZE for Upper Elementary• Measure of reading comprehension

• Comprehension is typically linked with phonics deficits

• Every 7-10 words in a passage, student chooses the best fit of 3 words in the context of what is being read• Correct, exact word from passage• Near Distracter – word of the same ‘type’ of word from passage (e.g. noun, verb, adverb)• Far Distracter – word not of the same type, selected

randomly from the passage• MAZE recommended grade 4 and beyond, better predictor

of future reading performance (Hosp, Hosp, & Howell, 2007).

• MAZE can be used as a supplemental measure [with ORF] to provide a more complete picture of a student’s reading skills (Shinn & Shinn, 2002)

McKenna (2010)

Outcomes Driven Model

Validate Needfor Support

ReviewOutcomes

EvaluateSupport

ImplementInstructional

Support

PlanInstructional

Support

Provide Instructional SupportBased on IntegratedAssessment - InterventionFeedback Loop

Identify Needfor Support

Good, Gruba, & Kaminski (2002)McKenna (2010)

Data Team Meetings • Reviews Universal Screening and Progress

Monitoring data – 3x’s per year for benchmark data review– at least once in between benchmark periods to review

progress monitoring data; monthly meetings, best practice when intervention programs are established

– align meeting schedule with decision-making guidelines for frequency of data review in your school district

• Data Team membership includes principal, literacy coach, school psychologist, special education teacher, ELL instructor, other as locally indicated

McKenna (2010)

Data Team Meetings • Options for meeting with grade-level teams(1)Grade level teams rotate throughout day

• Hire substitutes who will rotate with teachers• efficient, occurs in 1 day• Approximately 45 minute meetings

(2)Schedule end-of-day meeting across a week• 45 minute sessions, grade level teams organized by day

during the week• Schedule meeting times at the beginning of the

year! Plan effectively

McKenna (2010)

Data Team Meetings• Data team meets first, reviews data, and identifies

specific discussion targets and outcomes for specific grade levels

• Data team meets with Grade-Level teams – Identify successes through use of data – Identify skill areas where students are not performing

proficiently– Discuss potential adjustments that can be made to

increase instructional efficacy as realized by student gain; problem-solve challenges

– Review progress for concerns of specific students

McKenna (2010)

Required Data for Grouping• Summary of Effectiveness report per grade level

– Student movement from fall to winter– Student movement from winter to spring

• Summary of Effectiveness report for same skill • Class list reports • Individual student performance data from Universal

Screening and Progress Monitoring measures• Progress Monitoring data for students receiving

differentiated and targeted instruction and intervention• ELPA scores for students who are ELL• Grade-level teams bring in-program assessment data

McKenna (2010)

Summary of EffectivenessOverview

• Review student movement over time– fall to winter– winter to spring

• How do I read it? • Celebrate movement! Identify what worked to

move students forward and maintain gains• Review student regression: go deeper with other

reports and actual student performance data

McKenna (2010)

Summary of EffectivenessSystems Focus: Identify Precision Statements

• Identify precision statements to effectively answer questions from the data

• Precision statements have answers for who, what, when, where, and why– Who: What group of students– What: Skill area; specific skills– When: …and for how long does instruction occur?– Where: Does instruction occur in classroom or through pull-out

support? Is environment conducive to learning? – Why: What variables that we can control affect learning? How

can wrap-around support be provided to mediate other variables?

McKenna (2010)

Summary of EffectivenessSystems Focus: Questions to Answer

• Did 100% of students remain at benchmark?– How effective is the core program?

• What percentage of students moved out of a strategic range of support to benchmark?– What support did students receive? Differentiation through Core?

Intervention?

• What percentage of students moved out of an intensive range of support, performing at a strategic or intensive level?– How was their instructional program defined?

McKenna (2010)

Summary of EffectivenessSystems Focus: Questions to Answer

• What percentage of students did not remain at benchmark?– If over 20% moved back, review effectiveness of core program

and make adjustments• Professional Development?• Time, grouping, instruction?• Question to answer: How do we support program fidelity?

– Review individual student performance data to determine support that may be required

McKenna (2010)

Summary of EffectivenessSystems Focus: Questions to Answer

• What percentage of students moved from a strategic range of support to requiring intensive support?– How was instructional programming defined?– Assess fidelity of program implementation and

opportunity to learn– Review variables of time, grouping, and instruction

to increase program effectiveness– Creative problem-solving, ‘Think Outside the Box’

• LEAVE WITH A PLAN McKenna (2010)

Summary of Effectiveness

• Lets look at some data!

McKenna (2010)

Approaching Class Lists• Summary of Effectiveness is critical to review

student movement and performance on a systems-level – inform adjustments that need to be made within and

across grade levels to enhance core instruction– School-wide review of intervention programming

• Class lists and individual student performance data allows us to take closer look at where the break down is for students who are not demonstrating proficiency

McKenna (2010)

Guidelines for the Interpretation of Multiple Measures

• Identify the GOMs that represent pre-requisite skills in order to build a foundation for the attainment of other skills as aligned with the big ideas of beginning reading

• Review performance across all measures within the benchmark period • fall 2nd grade, NWF and ORF; mid-second through 5th, ORF

• Identify how discrepant a student is from the benchmark or normative data; validate concern• Discrepancy Ratio = Expected Performance /

Student Performance • Severely discrepant is 2x’s or greater• We want to intervene much earlier

• Magnitude of discrepancy and need for skill instruction indicates level of intensity of intervention required

McKenna (2010)

Guidelines for Multiple Measures• What level of support is required for students to be

successful?• Grade-Level Intervention / Walk to Read

– differentiate within core program – replacement core if enough students in grade level require

support to be successful• Specific Skill Instruction

– Reteaching, practice and repetition of skills not mastered – Phonics inventory to target skill instruction– Differentiation and/or intervention

• Widely Discrepant performance– Provide intensive support with explicit and systemic

intervention program– replacement core

McKenna (2010)

Questions to Answer• How effective is our Core program?

– are at least 80% of students responding to core instruction? • For which students is the core program effective and not

effective?– Strategic– Intensive– How discrepant is student performance?

• What skills need to be targeted for support?– Enhancement of Core program for all students– Small group skill instruction in core program– Supplemental program and Intervention

McKenna (2010)

In review of student performance for all…

• Identify specific populations of students– English Language Learners – Special Education– Review individual/group instructional programs for strength

in alignment of specificity of need within system of support

• Convergence of Evidence • Validate need for support

– do other measures of student performance also indicate skill deficit?

– Reassess student to confirm need for support and to more closely review patterns in student performance

McKenna (2010)

Universal Screening and Progress Monitoring Data

• Has the student/instructional group demonstrated gains in skill with progress monitoring?

• What is the current instructional program?– Is the student responding?

• Performance above the aim line (4 data points)? • Performance below the aimline (4 data points)? No growth? • Stable growth along the aimline (data hovers above and below)?

– Is the program being implemented as intended?

• Identify alterable variables of time, grouping and instruction to either intensify support, maintain support, or fade support/reintegrate

• Review data sources for Convergence of Evidence

McKenna (2010)

Second – Fifth Grade: Big Ideas

Students ‘not established’, requiring strategic support to make gains

• would benefit from small group instruction targeted to specific skill deficits that supports phonics skills

• in-classroom support beyond the core reading block may be indicated

• differentiate between higher and lower performance in strategic range

• use phonics inventory to identify specific skill sets for instruction

McKenna (2010)

Second – Fifth Grade: Big Ideas

Students ‘not established’, requiring strategic support to make gains

• progress monitor every other week if in the classroom; pull-out, progress monitor weekly (2x’s/wk, best practice)

• monitor accuracy and fluency – Look for patterns in performance

– The Devil’s in the details• Bottom line: It’s about phonics support.

McKenna (2010)

Second – Fifth Grade: Big Ideas

• Student’s requiring an intensive level of support to make gains– Identify median rate of accuracy across passages– Identify how discrepant a student is from his/her peers: * Discrepancy Ratio = performance goal/ student performance

• Student performance is considered severely discrepant if at or above 2 times discrepant

• Complete Phonics Inventory. Skill deficits are typically across the board

McKenna (2010)

Second – Fifth Grade: Big Ideas Student’s requiring an intensive level of support

to make gains• Typically, an explicit and systematic program will be

required to support meaningful gains• Progress Monitor Weekly

– 2x’s/wk, best practice– intensify progress monitoring when more data is needed

sooner; for example, the data team meets in a month to review progress for specific students

• Monitor accuracy and fluency – Look for patterns in performance

McKenna (2010)

Second – Fifth Grade: Big Ideas Low Accuracy, Low Rate of Oral Reading Fluency

• If student is widely discrepant (2x’s) consider administering a measure of NWF in order to determine is established in meeting the performance expectation of 50 letter sounds per minute and in the ability to recode/blend 15 CVC words.

• Assess if goal for PSF has been met (Phoneme Segmentation Fluency)• Provide remediation specific to skill deficit(s) via explicit

instruction through:– differentiation in the core or in a curriculum specific to phonics

development• Small Group instruction• Pre-teach lesson; target specific skill sets for repetition toward

mastery

McKenna (2010)

Second – Fifth Grade: Big Ideas Low Accuracy, Low Rate of Oral Reading Fluency

– Supplemental instruction through intervention beyond the core program (e.g. Phonics for Reading)

– Intensive Support through a core program that addresses the essential components of beginning reading (e.g. Reading Mastery, Corrective Reading)

– Additional opportunities for practice and repetition occur throughout the day to reinforce explicit skills taught (on the way to lunch, from recess, 5 minute wait before bus, check out before leave for activities and end of day)

McKenna (2010)

Second – Fifth Grade: Big Ideas Accurate, Low Rate of Oral Reading Fluency

• If student is widely discrepant, yet accurate, review existent error patterns.– Accurate, meaning at least a rate of 97% or higher– Student(s) may benefit from instruction in syllabication and/or

intervention that teaches an efficient approach to reading multisyllabic words.

– Assess if grade-level class has been taught skills of syllabication. If not, teach set to class. Otherwise, provide small group instruction within class.

– In addition to support for approaching multisyllabic words, build fluency through repeated reading program.

McKenna (2010)

Second Grade• Student’s not at benchmark, and for some who are,

the primary issue will be one of phonics• Identify skills that have been taught and that

students are expected to perform with classroom teacher and interventionist

• If students have not met benchmark for ORF, determine if they have met benchmark for NWF in the fall – review previous data for NWF and PSF

• Determine skill deficits through Phonics Screener and error analysis of NWF

McKenna (2010)

Second GradeStudents not meeting benchmark on NWF

• Consider degree of discrepancy• If widely discrepant (2x+) and multiple types of errors,

collect enough data to adequately identify problem 1) Define instructional program to meet level of

intensity of support needed– Is Reading Mastery required to support skill attainment? – Can support be met using Phonics for Reading?

2) Preteach or Reteach components of instructional program in which the student will participate

3) Prioritize and consider the most valuable use of instructional time; add instructional time for additional practice in most discrepant skill areas

McKenna (2010)

Second GradeStudents not meeting benchmark on NWF

• If students are not widely discrepant (< 2x’s), – analyze performance on NWF probe for accuracy, error patterns,

and blending (words recoded)

• Identify error patterns– vowel errors– decoding accurately, not blending– specific sound errors across nonsense words

• Provide small group instruction specific to error patterns identified, blending, or both (collect more data if unsure to validate problem identification)

• Pre-teach phonics elements of lesson in core program• Align intervention program to instructional need (see

Interventions at end of presentation)

McKenna (2010)

Second GradeStudents meeting benchmark on NWF in fall

not on ORF in Winter• Is the student reading with 97% accuracy or higher?

– if yes, reinforce skills initially mastered with additional practice, reteach error patterns that are identified, and build fluency

– if no, phonics screener and differentiate within class– depending on degree of discrepancy and breadth of skills for which

the student is not proficient, consider supplemental or intensive intervention program

• In a review of error patterns, had the skill been taught and is the student expected to perform it?

McKenna (2010)

Third-Fifth Grade• Reference notes for Grades 2-5, big ideas• If ORF is not been met in 3rd grade, consider

discrepancy from expected performance, complete phonics inventory– Has the goal for NWF been met?– Syllabication typically taught in 3rd grade

• If error patterns in 4th and 5th grade are primarily found with multisyllabic words, reteach to class/group in class– If error patterns are beyond that of multisyllabic words,

complete phonics inventory

McKenna (2010)

Vocabulary? Comprehension?• If students do not have skills and strategies to

approach reading text fluently, they will not be able to gain meaning from words or text

• Focus intervention on prerequisite skills in order to allow for independent access to content

• If a student reads with 97% accuracy or higher and would benefit from repeated reading, provide intervention beyond that of core reading instruction– Students with high accuracy, low fluency will frequently

need to re-read text to gain meaning – Repeated reading will assist to build fluency

McKenna (2010)

Vocabulary and Comprehension• Taught through anthology in core program• Schedule intervention groups to provide access to

instruction for vocabulary and comprehension in the core program – learners requiring intensive intervention

• Specific strategies for specific word instruction, word-learning strategies, and comprehension can be embedded in the core program to enhance it’s effectiveness

• Common language for use of skills and strategies should be cued and pre-corrected throughout content areas

McKenna (2010)

Vocabulary• Students requiring support beyond that of

differentiation within the classroom will require support in the area of phonics

• Embed in Core Instruction• Language for Thinking and Language for Writing

– Training required for fidelity– Particularly useful for students that see SLPs and

teachers of ELL

McKenna (2010)

Comprehension

• Comprehension Strategy Instruction– Focus on teaching 1 strategy at a time, well– Teach few, versus many, over time with depth– Background knowledge, summarization, comprehension

monitoring, sentence comprehension, story comprehension

• Soar to Success– Grades 3-8: comprehension strategies, reciprocal skills,

graphic organizers– Effective for Comprehension and Fluency

McKenna (2010)

Forming Instructional Groups• The goal is to identify what instructional program will work

for the student/group of students• Align intervention by need for skill instruction

– Differentiate in class or provide intervention– Many programs have placement tests

• Group size approximately 6 students – fewer students with more intensive need for support

• If support is being provided outside of the classroom, intervention time will be indicated by program. Most intervention programs will require at least 40 minutes of instructional time– Additional guidelines for instructional time in Carnine et al. (2006)

McKenna (2010)

Forming Instructional Groups• Often occurs with building principal and literacy

coach– it is helpful to coordinate scheduling with SPED

• All teachers have a copy of the intervention schedule

• Considerations for efficiency– Students walk to intervention room– progress monitor 1-2 students daily at end of lesson

McKenna (2010)

Considerations for the Interpretation of Spring Data

• Is spring data used as a formative or summative measure of student performance?

• What questions are being answered by the data?

• Will instructional groups shift this school year?– Does flexible grouping occur through scheduled

review of progress monitoring data?• Will tentative groups be formed for the fall?

McKenna (2010)

Evidence-Based Supplemental and Intervention Programs

http://oregonreadingfirst.uoregon.edu/http://www.fcrr.org/FCRRReports/index.aspx

McKenna (2010)

Supplemental and Intervention Programs versus Replacement Core

Supplementalto enhance core program for all studentsprevent/remediate skills for students in core who are

somewhat below grade level Intervention

prevent/remediate skills for students in core who are somewhat/significantly below grade level

Intervention to replace core for students who are substantially below grade level

Replacement core addresses all 5 big ideas of readingmore explicit instruction of finite skills, moves at

slower pace with some exceptionsMcKenna (2010)

Phonics • Explode the Code (1st – 3rd grade)

– Independent work; no research– Practice on explicit skills that have been taught

Phonics For Reading1st grade- 5th grade; Adult’s Learning English Languageuse for skill gaps, strategic supportinexpensive and easy to implementDaily 40-50 minutes or split lessons in 1/2

REWARDS◦ multisyllabic words◦ PLUS, focuses on Science and Social Studies

See Florida Center on Reading Research and Oregon Reading First websites for reviews of other programs for phonics supportMcKenna (2010)

Fluency• Read Naturally (50 words correct per minute to adult)

– Intervention program, not instruction– Primarily fluency-building; strong research– 97% accuracy or higher– 3, 30 minute lessons/week minimum

• Great Leaps (grades k-adult)– Fluency-only, easy to implement, 10-15 minutes

daily• Six Minute Solution (grades K-9)

– Derived from a strong research base; 3 levelsOnly 6 minutes of instructional time daily

McKenna (2010)

Replacement Core Programs• Reading Mastery Classic l, ll, lll (SRA) / Fast Cycle◦Modified orthography used in first level to emphasize

the specific sounds of letters◦Particularly powerful for children with significant

language as well as literacy deficits• Horizons (SRA), can include Funnix (computer-based)

◦ A, B; A/B covers 2 years in 1 to catch kids up to peers◦ Uses traditional orthography vs. Reading Mastery

• Corrective Reading for Decoding and Comprehension (SRA)–3rd -12th grade

McKenna (2010)

Replacement Core: Critical Points• Continue to teach skills of vocabulary and

comprehension from evidence-based core program • Not meant to be a long-term solution for large

groups of students – e.g. use Reading Mastery Fast Cycle to MOVE

students forward and to ACCELERATE their growth toward criterion and benchmark standards

• Use progress monitoring DATA to determine if students are gaining skills and generalizing them

• Define a plan for REINTEGRATION back into the core program (e.g. semester break in January)

McKenna (2010)

Instructional Grouping Take Home Points

• Use instructional recommendations as a guide; Do not go by instructional recommendation alone

• Look more closely at the data and specific skill areas of deficit and proficiency; be discerning

• Leave meetings with a plan for each grade level• Think creatively, outside of the box• Identify the smallest change that can be made to

make the biggest impact in student performanceMcKenna (2010)