ITEM ANALYSISARDHIAN SUSENO

CHOIRUL RISA PRADANA P.

ITEM ANALYSIS

is a process which examines student responses to individual test items (questions) in order to assess the quality of those items and of the test as a whole

SCOREPAK®

PURPOSE1. To improve items which will be used

again in later tests. 2. To eliminate ambiguous items in a

single test administration.3. To increase instructors' skills in test

construction.4. To identify specific areas of course

content which need greater emphasis or clarity.

Method 1) ITEM DIFFICULTY2) ITEM DISCRIMINATIO ITEM

CHARACTERISTIC CURVE3. Item validity (point biserial method)4. Effectiveness of distractors

I. ITEM DIFFICULTY• item difficulty is determined by the number of people

who answer a particular test item correctly (p).• For example, if the first question on a test was answered

correctly by 76% of the class, then the difficulty level (p or percentage passing) for that question is p = .76. If the second question on a test was answered correctly by only 48% of the class, then the difficulty level for that question is p = .48. The higher the percentage of people who answer correctly, the easier the item, so that a difficulty level of .48 indicates that question two was more difficult than question one, which had a difficulty level of .76.

Method of Item Difficulty

a) Method for Dichotomously Scored Item

b) Method for Polytomously Scored Item

c) Grouping Method

a). Method for Dichotomously Scored Item

Method for Dichotomously Scored Items

p is the difficulty of a certain item. R is the number of examinees who get that item

correct. N is the total number of examinees.

NRP

Example 1

There are 80 high school students attending a

science achievement test, and 61 students pass item

1. Please calculate the difficulty for item 1.

Answer : 0.7

b). Method for Polytomously Scored Items

maxXXP

X , the mean of total examinees’ scores on one item

maxX , the perfect scores of that item

Example 2 The perfect scores of one open- ended item is 20

points, the average score of total examinees on this

item is 11 points. What is the item difficulty?

Answer : 0.55

c). Grouping Method (Use of Extreme Groups)

• Upper (U) and Lower (L) Criterion groups are selected from the extremes of distribution of test scores or job ratings

2LU PPP

UP is the proportion for examinees of upper group who get the item correct.

LP is the proportion for examinees of lower group who get the item correct.

Example 3 There are 370 examinees attending a language

test. Known that 64 examinees of 27% upper

extreme group pass item 5, and 33 examinees of 27%

lower extreme group pass the same item. Please

compute the difficulty of item 5.

Answer : 0.49

II. ITEM DISCRIMINATION• Item discrimination refers to the degree to

which an item differentiates correctly among test takers in the behavior that the test is designed to measure.

• item discrimination determines whether those who did well on the entire test did well on a particular item.

Method of Item Discrimination

a). Index of Discrimination

D = pH - pL (7.5)

We need to set one or two cutting scores to divide the examinees into

upper scoring group and lower scoring group. pH is the proportion in the upper group who answer the item correctly and pL is the proportion in the lower group who answer

the item correctly. Values of D may range from -1.00 to 1.00.

ILLUSTRATION BY GGS INFORMATION SERVICES. CENGAGE LEARNING, GALE.

Another example

50 Examinees’ Test Data on 8-Item Scale About Job Stress.

Question: There are 140 students attending a world history test. (1)

If we use the ratio 27% to determine the upper and lower group, then how many examinees are there in the upper and lower group separately? (2)If 18 examinees in upper group answer item 5 correctly, and 6 examinees in lower group answer it correctly, then calculate the discrimination index for item 5.

ITEM VALIDITY :POINT-BISERIAL METHOD

ILLUSTRATION BY GGS INFORMATION SERVICES. CENGAGE LEARNING, GALE.

• Another way to determine the

discriminability of an item is to determine the correlation coefficient between performance on an item and performance on a test, or the tendency of students selecting the correct answer to have high overall scores. POINT-BISERIAL METHOD

Distractor (Incorrect Alternatives)

• Analyzing the distractors (e.i., incorrect alternatives) is useful in determining the relative usefulness of the decoys in each item. Items should be modified if students consistently fail to select certain multiple choice alternatives. The alternatives are probably totally implausible and therefore of little use as decoys in multiple choice items. A discrimination index or discrimination coefficient should be obtained for each option in order to determine each distractor's usefulness .