ABCs of CBMs

Summary of A Practical Guide to

Curriculum-Based Measurement

By: Michelle Hosp, John Hosp,

& Kenneth Howell

What will CBM provide?

CBM provides functional information required to inform educational decision making, including: Goal setting Planning Monitoring

What is CBM

CBM is composed of a set of standard directions, a timing device, a set of materials, scoring rules, standards for judging performance, and record forms or charts.

Attributes of CBM

1. Alignment Tested on the curriculum

2. Technically adequate Established reliability and validity Used to summarize both a student’s

level of performance and rate of progress

3. Makes use of criterion-referenced measures

Attributes of CBM cont.

4. Standard procedures are used Standard tasks are used for each

content Standard procedures are followed for

selecting or constructing testing materials

Standard administration and scoring directions exist for each procedure

5. Performance sampling is used (producing what is sometimes called behavioral or performance data)

Attributes of CBM cont.

6. Decision rules are put in place. These rules are based on performance criteria and are standardized through sampling or experimental procedures.

7. Emphasized repeated measurement over time. These repeated measures illustrate the degree to which an instructional intervention is producing learning.

Attributes of CBM cont.

8. Efficient. The measures can be given in a short period of time and quickly.

9. Data can be summarized efficiently. The results are summarized and interpreted as simple behavioral/performance statements.

How is CBM different?

CBM focuses on the curriculum and was designed to function within a problem-solving paradigm (e.g.

Responsiveness to Instruction RtI)

Differences? Alterable Variables - Designed to fill the need

for a data collection system that would produce the information required to guide instruction.

Low-inference Measure - CBM was not developed to explain how learning does or doesn’t occur, and it was not designed to conform to any particular theory about how students think, attend, remember, or process information. Therefore, inference and conjecture about what the resulting scores mean is kept to a minimum.

Differences? Criterion-references Measures - Used to

determine if students can demonstrate their knowledge by reaching specified performance levels on certain tasks. CBM came directly out of an intervention program. This means the measures must be: Aligned with curriculum Sensitive to instruction Repeatable so that progress monitoring could occur Able to determine if student has mastered task

How does CBM relate to RtI?

Key components of any good RtI approach are the use of

screening/benchmarking and progress-monitoring assessments.

CBMs are an efficient tool for meeting these components.

More about CBM CBM is not an instructional method or

intervention CBM is not a curriculum

The defining elements of CBM are the curriculum-based procedures for designing, administering, and scoring measures and for recording, summarizing, and interpreting the data that result from those measures.

A measure gets to be a CBM instrument if it is designed, administered, and scored according to established CBM procedures.

Different types of CBM

General Outcome Measures (GOMs) Used to sample performance across several goals at

the same time by using capstone tasks that are complex in the sense that they can only be accomplished by successfully applying a number of contributing skills.

Advantages Efficient by measuring several subskills Recognizes the limitations of isolating subskills from the

context in which they normally are expected to function Visual displays of progress will show longer acquisition

slopes, allowing adequate opportunities for progress monitoring and data-based instructional modifications

Disadvantage May need more diagnostic information

Example - Oral Reading Fluency

Different types of CBM cont. Skills-based Measures

Designed to accomplish many of the functions of GOMs

Advantages Will produce long acquisition slopes like GOMs Can yield some analytical information as long as

steps are taken to ensure that an adequate sample of each kind of item is provided and that the items are cross-referenced to goals.

Disadvantage At any given time only a few items on the test will

be directly related to what the student is currently learning.

Example - Math Computation

Different types of CBMs cont.

Mastery Measures Used on parts of the curriculum that contain discrete

and easily identified sets (or domains) of items that are closely related by some common skill, theme concept, or solution strategy.

Advantages Focuses on a particular set of skills. Can help troubleshoot a problem and need to do

specific-level testing Monitor learning when a skill is being taught in isolation

Disadvantage Not good for surveying general levels of performance or

for monitoring growth on long-term goals. Example - Math Facts

CBMs & the Curriculum

What is the definition of curriculum? Curriculum is “what you teach” not

“how you teach.” Therefore, you do not need to have

assessments using the same formats and examples as the instructional materials.

Reasons for this position Instructional programs do not follow

the same sequences and schedules.

Program-specific tests may not tell you if the learning has generalized. (You do not want a student who can only work problems that are presented in a certain format).

Program-specific tests will make the teacher dependent on the program.

CBMs can be used for…

Screening/benchmarking decisions Progress-monitoring decisions Diagnostic decisions Outcome decisions

CBM is most appropriate for screening/benchmarking decisions and progress-monitoring decisions.

Where do CBMs get the performance standards? Exemplar Sampling - Draw a sample of

successful students for the skill in question. Norm Sampling - Testing a large group of

students and using the average scores at a grade level or some percentage of them as the target level of performance for that grade.

Predictive Validity or Benchmarks - Involves empirically or statistically determining the level of performance that reliably predicts successful performance on a different outcome measure, usually at a future date.

AIMSweb uses norm sampling and benchmarks.

CBM examples

Reading CBM (Oral Reading Fluency) Reading CBM provides a reliable

and valid way to:1. Identify students who are at risk for

reading failure2. Identify which students are not

making adequate progress given the instruction they are receiving

3. Identify students’ instructional level4. Identify which students need

additional diagnostic assessment

Reading CBM (Oral Reading Fluency) Automaticity is important because it

demonstrates the student has mastered the skill.

In order to read a passage of text aloud quickly and accurately, you need to use a variety of different literacy skills, including decoding, vocabulary, and comprehension (particularly accessing prior knowledge).

This is what makes ORF a good predictor of future reading performance.

FAQ about Oral Reading Fluency

Should I have students read the title on the page or should I it to them? You should not have the student read the title nor

should you read it to the student. It is not counted in the total words for the passage and may give the students some background information. Remember, this is a time to assess, not teach.

Can the benchmark scores be used to put students in instructional groups? Yes, if you have students with similar instructional

needs. These groups should be flexible, and students should be evaluated and regrouped every 6-8 weeks.

Other questions can be found in the book.

MAZE

A measure that looks at comprehension.

Early Literacy

These measures require the student to be accurate as well as fluent allowing one to determine how automatic students are at performing a task.

Remember, automaticity is important because it represents a higher level of skill mastery.

Early Literacy Measures Initial sound fluency and phoneme

segmentation fluency are the two measures designed to assess phonological awareness.

Nonsense word fluency is a measure designed to assess decoding skills.

Letter naming fluency is used as a risk indicator for reading.

Word identification fluency is a measure of phonics and fluency.

Spelling

Spelling is an important skill for writing and evaluating that also provides information about a student’s decoding.

Good spellers are always good readers, but the reverse is not always true; a poor speller can be a good reader or a poor reader.

FAQ about Spelling CBM Can I give students the list prior to the assessment?

No. This assessment is a general measure of students’ skill at applying the spelling rules they have been previously taught. Giving students the list prior to the assessment increases the chance that students will attempt to memorize the words instead of learning the spelling skills being taught.

The list for this week doesn’t include the spelling rules I just taught. Can I choose one that does? No. Avoid teaching to the test and remember Spelling CBM is

designed to assess general, not specific, skill mastery. Should I teach students the spelling rules they miss on the

spelling lists? Yes. Whenever students are missing previously taught rules,

it is beneficial to reteach the material. This may be an indication that students did not fully understand the concept. Teachers should, however, continue to use the scope and sequence presented in the curriculum.

Other questions can be found in the book.

Writing Several components of written

communication pertinent to student progress in written language include: Writing fluency, syntactic maturity,

vocabulary or semantic maturity, content, and conventions.

Numerous studies have presented CBMs available for many of these target areas, including: Total words written, words spelled correctly,

correct writing sequences, and total correct punctuation.

Math Reason for using a math CBM

Most math assessments do not provide information about fluency, which provides information about skill mastery, but CBM does.

Single-skill math sheets are not appropriate for screening or progress monitoring (because it only assesses one skill), but it can be used to gain some diagnostic information or to use as a starting point for a curriculum-based evaluation approach to decision making.

When scoring Math CBM, the number of correct digits in the solution to the problem is used because it is a more sensitive measure to change.

Charting and Graphing CBM Data to Help Make Decisions

Three methods for setting goals

End-of-Year Benchmarks This is the lowest score we would

accept that would indicate a student is not at risk for future academic failure.

Three methods for setting goals, cont. Norms

When available, national norms are good to use.

Come in two forms:1. Levels of Performance: Use same process as

with end-of-year benchmarks. The difference is that these are based on typical performance of same-grade peers rather than a criterion for proficiency that predicts performance on outcome measures.

2. Rate of growth or progress: This can be calculated as an average weekly gain.

Three methods for setting goals, cont. Intraindividual Framework

Uses the student’s current level of performance and rate of progress to set end-of-the-year goals.

It may underestimate a student’s rate of learning and may never catch the student up if the student started behind.

Graphing

To graph, identify a specific target to work toward and number of weeks it will take to get there.

Decision Rules

Two methods are used for making decisions:

1. Data point analysis - The data points on the graph for each week are examined.

2. Trend line analysis - Represents the student’s observed rate of progress, which can be compared to the expected rate of progress as indicated by the goal line.

How to use CBM in RtI The same CBM measures can be used with

all students across the various tiers of instruction.

Types of decisions1. Effectiveness of the instructional program that a

student is receiving. CBM does not give you information specific enough to determine which instructional approach to use or how to alter the current one - this is a time for teachers’ professional judgment.

2. Eligibility for special education. The most common approach of eligibility is called the dual discrepancy method. CBM provides you with performance data and progress data. You can compare both of these types of data to standards.

How to learn more

You can learn more about CBMs by reading the book, The ABCs of CBM.

Reviewing the website: National center on Student Progress Monitoring: www.studentprogress.org