educational assessment of students 524 - january 2020/textbook... · z scores •communicate...

Educational Assessment of Students7e

Susan M. Brookhart & Anthony J. Nitko

Copyright © 2015, 2012, 2009 by Pearson Education, Inc. All Rights Reserved

Chapter 17

Interpreting Norm-Referenced Scores

Educational Assessment of Students, 7eBrookhart & Nitko

Copyright © 2015, 2012, 2009 by Pearson Education, Inc.

All Rights Reserved

Key Concept #1

• A referencing framework is a structure used to compare a student’s performance to something external to the assessment in order to interpret performance.

A norm-referencing framework interprets a student’s assessment performance by comparing it to the performance of a well-defined group of other students who have taken the same assessment.

A criterion-referencing framework interprets a student’s performance according to the kinds of performances a student can do in a domain.

A standards-referenced framework combines elements of both.



All Rights Reserved

Key Concept #2

• Use normative information to describe student strengths, weaknesses, and progress.



All Rights Reserved

Key Concept #3

• Test publishers may provide norm-referenced scores based on information from several different norm groups.



All Rights Reserved

Norm Groups

• Local norm group: students in the same grade in the same school district

schools/districts should have scores

publishers may offer scores

• National norm group: intended to be representative of students in the country

test publishers use different norming procedures

• Special norm group



All Rights Reserved

Norm Groups

• School averages norms: ranked tabulation of the average (mean) score from each school building in a national sample of schools



All Rights Reserved

Guidelines for Using Publishers’ Norms

• Make sure the norm group is:

relevant

representative

recent



All Rights Reserved

Key Concept #4

• Different types of norm-referenced scores are constructed to serve different purposes.



All Rights Reserved

Key Concept #5

• The percentile rank tells the percentage of the students in a norm group who have scored lower than the raw score in question.



All Rights Reserved

Percentile Ranks: Advantages

• Easily understood.

• Clearly reflect norm-referencing.

• Permit a person’s performance to be compared to a variety of norm groups.

• Can be used to compare a student’s relative standing in each of several achievement or ability areas.



All Rights Reserved

Percentile Ranks: Limitations

• Can be confused with percentage correct scores.

• Can be confused with some other types of two-digit derived scores.

• Do not form an equal-interval scale.



All Rights Reserved

Key Concept #6

• A linear standard score tells how far a raw score is from the mean of the norm group, expressing the distance in standard deviation units.



All Rights Reserved

Z Scores

• Communicate students’ norm-referenced achievement expressed as a distance away from the mean.

if Z = -1.5, the student’s score is 1.5 standard deviations below the average score.

• Can be used for norm-referenced comparison of raw scores with different metrics



All Rights Reserved

SS Scores

• Tells the location of a raw score in a distribution having a mean of 50 and a standard deviation of 10.

transformation of z score

if Z = -1.5, SS = 35



All Rights Reserved

Key Concept #7

• A normal distribution is a mathematical model (an equation) based on the mean and standard deviation of a set of scores.

Normal curves are smooth, continuous, symmetrical, and bell shaped.



All Rights Reserved

Key Concept #8

• Normalized standard scores are based on transforming raw scores on an assessment to make them fit a normal distribution.



All Rights Reserved

Normalized z -scores

• z-scores that have percentile ranks corresponding to what we would expect in a normal distribution

Raw Score Percentile rank Normalized

standard (zn)

Linear

standard (z)

36 98 2.05 2.43

33 96 1.75 1.64

15 4 -1.75 -3.09

14 2 -2.05 -3.36



All Rights Reserved

Normalized T-scores

• Tell the location of a raw score in a normal distribution having a mean of 50 and a standard deviation of 10. Joey’s T -score is 40, which means he is one

standard deviation below the mean of the norm group, and his percentile rank is approximately 16.



All Rights Reserved

Deviation IQ Scores

• Tells the location of a raw score in a normal distribution having a mean of 100 and a standard deviation of 15 or 16.

Meghan has DIQ = 116, which means she has scored one standard deviation above the mean of her age group and the percentile rank of her score is 84.



All Rights Reserved

Stanines

• Tells the location of a raw score in a specific segment of a normal distribution.

Blake’s stanine on the spelling subtest of the standardized test was 3, which means that his raw score was in the lower 20% of the norm group. Specifically, his percentile rank was between 11 and 22.



All Rights Reserved

SAT-Scores

• Historically, a normalized standard score from a distribution that has a mean of 500 and a standard deviation of 100.

No longer use this computation, but the current scores on the 200-800 scale can be compared across administrations



All Rights Reserved

Normal Curve Equivalents

• Normalized standard score with a mean of 50 and a standard deviation of 21.06.

Primary value is evaluating gains from various educational programs that use different publishers’ tests



All Rights Reserved

Key Concept #9

• Developmental and educational growth scales are norm-referenced scores that can be used to chart educational development or progress.

extended normalized standard score scale

grade-equivalent score scale



All Rights Reserved

Key Concept #10

• An extended normalized standard score tells the location of a raw score on a scale that is anchored to a lower grade reference group.

based on extended z scale OR

based on item response theory



All Rights Reserved

Key Concept #11

• A grade-equivalent score tells the grade level at which a raw score is average.

Useful for reporting educational development.

Provided by test publisher.

• GE is the median score (sometimes mean score) in each grade’s norm group.

• A third grader’s GE score of 5.7 on a mathematics test covering third-grade content does not mean that this student should be placed in fifth-grade math.



All Rights Reserved

Things to Keep in Mind When Interpreting Grade Equivalents

• In some subject areas, students’ performance drops over the summer months.

• The meaning of grade-equivalent scores for a subject depends very much on the subject matter.

• Grade-equivalent scores do not necessarily indicate mastery of the material.

• The more closely the test items match the material emphasized in the classroom before the test was administered, the more likely the students will score well above grade level.



All Rights Reserved

Things to Keep in Mind When Interpreting Grade Equivalents

• Grade equivalents from different tests cannot be interchanged.

• Grade equivalents for different subjects cannot be compared.

• Grade equivalents do not indicate “normal” growth.

• The grade-equivalent score scale does not have a one-to-one correspondence with the number of questions a student answers correctly on a test.

• Grade mean equivalents are also problematic.



All Rights Reserved

Key Concept #12

• Five guidelines for score interpretation will serve you well:

Look for patterns in scores.

Seek explanations for the patterns.

Don’t expect many surprises.

Don’t overinterpret small differences.

Use evidence from other assessments to clarify interpretations.



All Rights Reserved

Typical Misunderstandings

• The grade-equivalent score tells which grade the student should be in.

THEY DO NOT.

• The percentile rank and percent-correct scores mean the same thing.

THEY DO NOT.

• The percentile rank norm group consists of only the students in a particular classroom.

IT DOES NOT.

• “Average” is the standard to beat.

IT IS NOT.



All Rights Reserved

Typical Misunderstandings

• Small changes in percentile ranks over time are meaningful.

THEY ARE NOT.

• Percent-correct scores below 70 are failing.

THEY ARE (USUALLY) NOT.

• If you get a perfect score, your percentile rank must be 99.

IT MAY NOT BE.

educational assessment of students 524 - january 2020/textbook... · z scores •communicate...

Documents