![Page 1: Estimation of Ability Using Globally Optimal Scoring Weights Shin-ichi Mayekawa Graduate School of Decision Science and Technology Tokyo Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/5697bf7a1a28abf838c8310f/html5/thumbnails/1.jpg)
Estimation of Ability Using
Globally Optimal Scoring Weights
Shin-ichi Mayekawa
Graduate School of Decision Science and Technology
Tokyo Institute of Technology
![Page 2: Estimation of Ability Using Globally Optimal Scoring Weights Shin-ichi Mayekawa Graduate School of Decision Science and Technology Tokyo Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/5697bf7a1a28abf838c8310f/html5/thumbnails/2.jpg)
2OutlineReview of existing methodsGlobally Optimal Weight: a set of
weights that maximizes the Expected Test Information
Intrinsic Category WeightsExamplesConclusions
![Page 3: Estimation of Ability Using Globally Optimal Scoring Weights Shin-ichi Mayekawa Graduate School of Decision Science and Technology Tokyo Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/5697bf7a1a28abf838c8310f/html5/thumbnails/3.jpg)
3BackgroundEstimation of IRT ability on the basis of
simple and weighted summed score X.Conditional distribution of X given
as the distribution of the weighted sum of the Scored Multinomial Distribution.
Posterior Distribution of given X.
h(x) f(x|) h()Posterior Mean(EAP) of given X.Posterior Standard Deiation(PSD)
![Page 4: Estimation of Ability Using Globally Optimal Scoring Weights Shin-ichi Mayekawa Graduate School of Decision Science and Technology Tokyo Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/5697bf7a1a28abf838c8310f/html5/thumbnails/4.jpg)
4Item Score
We must choose w to calculate X.
IRF
![Page 5: Estimation of Ability Using Globally Optimal Scoring Weights Shin-ichi Mayekawa Graduate School of Decision Science and Technology Tokyo Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/5697bf7a1a28abf838c8310f/html5/thumbnails/5.jpg)
5Item Score
We must choose w and v to calculate X.
ICRF
![Page 6: Estimation of Ability Using Globally Optimal Scoring Weights Shin-ichi Mayekawa Graduate School of Decision Science and Technology Tokyo Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/5697bf7a1a28abf838c8310f/html5/thumbnails/6.jpg)
6Conditional distribution of X given Binary items
Conditional distribution of summed score X.Simple sum: Walsh(1955), Lord(1969)Weighted sum: Mayekawa(2003)
Polytomous itemsConditional distribution of summed score X.
Simple sum: Hanson(1994), Thissen et.al.(1995)
With Item weight and Category weight: Mayekawa & Arai(2007)
![Page 7: Estimation of Ability Using Globally Optimal Scoring Weights Shin-ichi Mayekawa Graduate School of Decision Science and Technology Tokyo Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/5697bf7a1a28abf838c8310f/html5/thumbnails/7.jpg)
7ExampleEight Graded Response Model items
3 categories for each item.
![Page 8: Estimation of Ability Using Globally Optimal Scoring Weights Shin-ichi Mayekawa Graduate School of Decision Science and Technology Tokyo Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/5697bf7a1a28abf838c8310f/html5/thumbnails/8.jpg)
8Example (choosing weight)Example: Mayekawa and Arai (2008)
small posterior variance good weight. Large Test Information (TI) good weight
![Page 9: Estimation of Ability Using Globally Optimal Scoring Weights Shin-ichi Mayekawa Graduate School of Decision Science and Technology Tokyo Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/5697bf7a1a28abf838c8310f/html5/thumbnails/9.jpg)
9Test Information Function Test Information Function is proportional
to the slope of the conditional expectation of X given (TCC), and inversely proportional the squared width of the confidence interval (CI) of given X.
Width of CIInversely proportional to the conditional
standard deviation of X given .
![Page 10: Estimation of Ability Using Globally Optimal Scoring Weights Shin-ichi Mayekawa Graduate School of Decision Science and Technology Tokyo Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/5697bf7a1a28abf838c8310f/html5/thumbnails/10.jpg)
10Confidence interval (CI) of given X
![Page 11: Estimation of Ability Using Globally Optimal Scoring Weights Shin-ichi Mayekawa Graduate School of Decision Science and Technology Tokyo Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/5697bf7a1a28abf838c8310f/html5/thumbnails/11.jpg)
11Test Information Functionfor Polytomous Items
ICRF
![Page 12: Estimation of Ability Using Globally Optimal Scoring Weights Shin-ichi Mayekawa Graduate School of Decision Science and Technology Tokyo Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/5697bf7a1a28abf838c8310f/html5/thumbnails/12.jpg)
12Maximization of the Test Informationwhen the category weights are known.Category weighted Item Score
and the Item Response Function
![Page 13: Estimation of Ability Using Globally Optimal Scoring Weights Shin-ichi Mayekawa Graduate School of Decision Science and Technology Tokyo Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/5697bf7a1a28abf838c8310f/html5/thumbnails/13.jpg)
13Maximization of the Test Informationwhen the category weights are known.
![Page 14: Estimation of Ability Using Globally Optimal Scoring Weights Shin-ichi Mayekawa Graduate School of Decision Science and Technology Tokyo Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/5697bf7a1a28abf838c8310f/html5/thumbnails/14.jpg)
14Maximization of the Test Informationwhen the category weights are known.Test Information
![Page 15: Estimation of Ability Using Globally Optimal Scoring Weights Shin-ichi Mayekawa Graduate School of Decision Science and Technology Tokyo Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/5697bf7a1a28abf838c8310f/html5/thumbnails/15.jpg)
15Maximization of the Test Informationwhen the category weights are known.First Derivative
![Page 16: Estimation of Ability Using Globally Optimal Scoring Weights Shin-ichi Mayekawa Graduate School of Decision Science and Technology Tokyo Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/5697bf7a1a28abf838c8310f/html5/thumbnails/16.jpg)
16Maximization of the Test Informationwhen the category weights are known.
![Page 17: Estimation of Ability Using Globally Optimal Scoring Weights Shin-ichi Mayekawa Graduate School of Decision Science and Technology Tokyo Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/5697bf7a1a28abf838c8310f/html5/thumbnails/17.jpg)
17Globally Optimal WeightA set of weights that maximize
the Expected Test Informationwith some reference distribution of .
It does NOT depend on .
![Page 18: Estimation of Ability Using Globally Optimal Scoring Weights Shin-ichi Mayekawa Graduate School of Decision Science and Technology Tokyo Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/5697bf7a1a28abf838c8310f/html5/thumbnails/18.jpg)
18Example
NABCT A B1 B2 GO GOINT A AINT Q1 1.0 -2.0 -1.0 7.144 7 8.333 8Q2 1.0 -1.0 0.0 7.102 7 8.333 8Q3 1.0 0.0 1.0 7.166 7 8.333 8Q4 1.0 1.0 2.0 7.316 7 8.333 8Q5 2.0 -2.0 -1.0 17.720 18 16.667 17Q6 2.0 -1.0 0.0 17.619 18 16.667 17Q7 2.0 0.0 1.0 17.773 18 16.667 17Q8 2.0 1.0 2.0 18.160 18 16.667 17
LOx LO GO GOINT A AINT CONST 7.4743 7.2993 7.2928 7.2905 7.2210 7.2564 5.9795
![Page 19: Estimation of Ability Using Globally Optimal Scoring Weights Shin-ichi Mayekawa Graduate School of Decision Science and Technology Tokyo Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/5697bf7a1a28abf838c8310f/html5/thumbnails/19.jpg)
19Maximization of the Test Informationwith respect to the category weights.Absorb the item weight in category
weights.
![Page 20: Estimation of Ability Using Globally Optimal Scoring Weights Shin-ichi Mayekawa Graduate School of Decision Science and Technology Tokyo Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/5697bf7a1a28abf838c8310f/html5/thumbnails/20.jpg)
20Maximization of the Test Informationwith respect to the category weights.Test Information
Linear transformation of the categoryweights does NOT affect the information.
![Page 21: Estimation of Ability Using Globally Optimal Scoring Weights Shin-ichi Mayekawa Graduate School of Decision Science and Technology Tokyo Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/5697bf7a1a28abf838c8310f/html5/thumbnails/21.jpg)
21Maximization of the Test Informationwith respect to the category weights.First Derivative
![Page 22: Estimation of Ability Using Globally Optimal Scoring Weights Shin-ichi Mayekawa Graduate School of Decision Science and Technology Tokyo Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/5697bf7a1a28abf838c8310f/html5/thumbnails/22.jpg)
22Maximization of the Test Informationwith respect to the category weights.Locally Optimal Weight
![Page 23: Estimation of Ability Using Globally Optimal Scoring Weights Shin-ichi Mayekawa Graduate School of Decision Science and Technology Tokyo Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/5697bf7a1a28abf838c8310f/html5/thumbnails/23.jpg)
23Globally Optimal WeightWeights that maximize
the Expected Test Informationwith some reference distribution of .
![Page 24: Estimation of Ability Using Globally Optimal Scoring Weights Shin-ichi Mayekawa Graduate School of Decision Science and Technology Tokyo Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/5697bf7a1a28abf838c8310f/html5/thumbnails/24.jpg)
24Intrinsic category weightA set of weights which maximizes:
Since the category weights can belinearly transformed, we set v0=0, ….. vmax=maximum item score.
![Page 25: Estimation of Ability Using Globally Optimal Scoring Weights Shin-ichi Mayekawa Graduate School of Decision Science and Technology Tokyo Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/5697bf7a1a28abf838c8310f/html5/thumbnails/25.jpg)
25Example of Intrinsic Weights
![Page 26: Estimation of Ability Using Globally Optimal Scoring Weights Shin-ichi Mayekawa Graduate School of Decision Science and Technology Tokyo Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/5697bf7a1a28abf838c8310f/html5/thumbnails/26.jpg)
26Example of Intrinsic Weightsh()=N(-0.5, 1): v0=0, v1=*, v2=2
![Page 27: Estimation of Ability Using Globally Optimal Scoring Weights Shin-ichi Mayekawa Graduate School of Decision Science and Technology Tokyo Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/5697bf7a1a28abf838c8310f/html5/thumbnails/27.jpg)
27Example of Intrinsic Weightsh()=N(0.5, 1): v0=0, v1=*, v2=2
![Page 28: Estimation of Ability Using Globally Optimal Scoring Weights Shin-ichi Mayekawa Graduate School of Decision Science and Technology Tokyo Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/5697bf7a1a28abf838c8310f/html5/thumbnails/28.jpg)
28Example of Intrinsic Weightsh()=N(1, 1 ): v0=0, v1=*, v2=2
![Page 29: Estimation of Ability Using Globally Optimal Scoring Weights Shin-ichi Mayekawa Graduate School of Decision Science and Technology Tokyo Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/5697bf7a1a28abf838c8310f/html5/thumbnails/29.jpg)
29Summary of Intrinsic WeightIt does NOT depend on , but
depends on the reference distributionof : h() as follows.
For the 3 category GRM, we found thatFor those items with high discrimination
parameter, the intrinsic weights tendto become equally spaced: v0=0, v1=1, v2=2
The Globally Optimal Weight isnot identical to the Intrinsic Weights.
![Page 30: Estimation of Ability Using Globally Optimal Scoring Weights Shin-ichi Mayekawa Graduate School of Decision Science and Technology Tokyo Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/5697bf7a1a28abf838c8310f/html5/thumbnails/30.jpg)
30Summary of Intrinsic WeightFor the 3 category GRM, we found that
The mid-category weight v1 increases according to the location of the peak ofICRF. That is:
The more easy the category is,
the higher the weight .
v1 is affected by the relative location ofother two category ICRFs.
![Page 31: Estimation of Ability Using Globally Optimal Scoring Weights Shin-ichi Mayekawa Graduate School of Decision Science and Technology Tokyo Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/5697bf7a1a28abf838c8310f/html5/thumbnails/31.jpg)
31Summary of Intrinsic WeightFor the 3 category GRM, we found that
The mid-category weight v1 decreases according to the location of the reference distribution of h()
If the location of h() is high, the mostdifficult category gets relatively high weight,and vice versa.
When the peak of the 2nd categorymatches the mean of h(), we haveeqaully spaced category weights:
v0=0, v1=1, v2=2
![Page 32: Estimation of Ability Using Globally Optimal Scoring Weights Shin-ichi Mayekawa Graduate School of Decision Science and Technology Tokyo Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/5697bf7a1a28abf838c8310f/html5/thumbnails/32.jpg)
32Globally Optimal w given v
![Page 33: Estimation of Ability Using Globally Optimal Scoring Weights Shin-ichi Mayekawa Graduate School of Decision Science and Technology Tokyo Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/5697bf7a1a28abf838c8310f/html5/thumbnails/33.jpg)
33Test Information
LOx LO GO GOINT CONST 30.5320 30.1109 30.0948 29.5385 24.8868
![Page 34: Estimation of Ability Using Globally Optimal Scoring Weights Shin-ichi Mayekawa Graduate School of Decision Science and Technology Tokyo Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/5697bf7a1a28abf838c8310f/html5/thumbnails/34.jpg)
34Test Information
![Page 35: Estimation of Ability Using Globally Optimal Scoring Weights Shin-ichi Mayekawa Graduate School of Decision Science and Technology Tokyo Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/5697bf7a1a28abf838c8310f/html5/thumbnails/35.jpg)
35Bayesian Estimation of from X
![Page 36: Estimation of Ability Using Globally Optimal Scoring Weights Shin-ichi Mayekawa Graduate School of Decision Science and Technology Tokyo Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/5697bf7a1a28abf838c8310f/html5/thumbnails/36.jpg)
36Bayesian Estimation of from X
![Page 37: Estimation of Ability Using Globally Optimal Scoring Weights Shin-ichi Mayekawa Graduate School of Decision Science and Technology Tokyo Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/5697bf7a1a28abf838c8310f/html5/thumbnails/37.jpg)
37Bayesian Estimation of from X
(1/0.18)^2 = 30.864
![Page 38: Estimation of Ability Using Globally Optimal Scoring Weights Shin-ichi Mayekawa Graduate School of Decision Science and Technology Tokyo Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/5697bf7a1a28abf838c8310f/html5/thumbnails/38.jpg)
38ConclusionsPolytomous item has the Intrinsic
Weight.By maximizing the Expected Test
Information with respect to either Item or Category weights, we can calculate the Globally Optimal Weights which do not depend on .
Use of the Globally Optimal Weights when evaluating the EAP of given X reduces the posterior variance.
![Page 39: Estimation of Ability Using Globally Optimal Scoring Weights Shin-ichi Mayekawa Graduate School of Decision Science and Technology Tokyo Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/5697bf7a1a28abf838c8310f/html5/thumbnails/39.jpg)
39References
![Page 40: Estimation of Ability Using Globally Optimal Scoring Weights Shin-ichi Mayekawa Graduate School of Decision Science and Technology Tokyo Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/5697bf7a1a28abf838c8310f/html5/thumbnails/40.jpg)
40
ご静聴ありがとうございました。
Thank you.
![Page 41: Estimation of Ability Using Globally Optimal Scoring Weights Shin-ichi Mayekawa Graduate School of Decision Science and Technology Tokyo Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/5697bf7a1a28abf838c8310f/html5/thumbnails/41.jpg)
41
![Page 42: Estimation of Ability Using Globally Optimal Scoring Weights Shin-ichi Mayekawa Graduate School of Decision Science and Technology Tokyo Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/5697bf7a1a28abf838c8310f/html5/thumbnails/42.jpg)
42
![Page 43: Estimation of Ability Using Globally Optimal Scoring Weights Shin-ichi Mayekawa Graduate School of Decision Science and Technology Tokyo Institute of Technology](https://reader035.vdocument.in/reader035/viewer/2022062423/5697bf7a1a28abf838c8310f/html5/thumbnails/43.jpg)
43