shojima kojiro the national center for university entrance examinations [email protected]
DESCRIPTION
Asymmetric Triangulation Scaling: An asymmetric MDS for extracting inter-item dependency structure from test data. SHOJIMA Kojiro The National Center for University Entrance Examinations [email protected]. Purpose of Research. - PowerPoint PPT PresentationTRANSCRIPT
Asymmetric Triangulation Scaling:An asymmetric MDS for extracting
inter-item dependency structure from test data
SHOJIMA KojiroThe National Center for University Entrance Examinations
Purpose of Research
• Development of method for visualizing inter-item dependency structure– Especially important for analyzing math test data
• Proposal: ATRISCAL– Asymmetric TRIangulation SCALing
– An asymmetric multidimensional scaling
– Conditional correct response rate matrix is object of analysis
Joint correct response rate matrix
• n×n symmetry matrix• The j-th diagonal element P(j,j)=P(j) – Correct response rate of item j
• The ij-th off-diagonal element P(i,j) – Joint correct response rate of items i and j– symmetry P(i,j)=P(j,i)
Item 1 Item 2 ⋯ Item nItem 1 P(1,1) P(1,2) ⋯ P(1,n)Item 2 P(2,1) P(2,2) ⋯ P(2,n)⋮ ⋮ ⋮ ⋱ ⋮
Item n P(n,1) P(n,2) ⋯ P(n,n)
Conditional correct response rate matrix
• n×n asymmetry matrix• The j-th diagonal element P(j|j)=P(j)/P(j)=1.0• The ij-th off-diagonal element P(j|i)=P(i,j)/P(i)– The correct response rate of item j when item i is
answered correctly– P(i|j)≠P(j|i): Usually asymmetric
Item 1 Item 2 ⋯ Item nItem 1 P(1,1) P(1,2) ⋯ P(1,n)Item 2 P(2,1) P(2,2) ⋯ P(2,n)⋮ ⋮ ⋮ ⋱ ⋮
Item n P(n,1) P(n,2) ⋯ P(n,n)
Item 1 Item 2 ⋯ Item nItem 1 P(1,1)/P(1) P(1,2)/P(1) ⋯ P(1,n)/P(1)Item 2 P(2,1)/P(2) P(2,2)/P(2) ⋯ P(2,n)/P(2)⋮ ⋮ ⋮ ⋱ ⋮
Item n P(n,1)/P(n) P(n,2)/P(n) ⋯ P(n,n)/P(n)
Item 1 Item 2 ⋯ Item nItem 1 1 P(2|1) ⋯ P(n|1)Item 2 P(1|2) 1 ⋯ P(n|2)⋮ ⋮ ⋮ ⋱ ⋮
Item n P(1|n) P(2|n) ⋯ 1
Multidimensional scaling (MDS)
X12
X3X9
X7
X13
X10
X8
X11
X2
X4
X6
X5
X14
X1
X15
Q1
QM Q2
O
Asymmetry conditional correct response rate matrixItem 1 Item 2 ⋯ Item n
Item 1 1 P(2|1) ⋯ P(n|1)Item 2 P(1|2) 1 ⋯ P(n|2)⋮ ⋮ ⋮ ⋱ ⋮
Item n P(1|n) P(2|n) ⋯ 1
𝑂𝑋𝑖𝑗ሬሬሬሬሬሬሬሬԦ⊥𝑋𝑖𝑋𝑗ሬሬሬሬሬሬሬሬԦ
𝑂𝑋𝑖𝑗ሬሬሬሬሬሬሬሬԦ= 𝑘𝑂𝑋𝑖ሬሬሬሬሬሬሬԦ+ሺ1− 𝑘ሻ𝑂𝑋𝑗ሬሬሬሬሬሬሬԦ (0 ≤ 𝑘 ≤ 1)
|𝑂𝑋𝑖ሬሬሬሬሬሬሬԦ| = 𝑃(𝑖) |𝑂𝑋𝑗ሬሬሬሬሬሬሬԦ| = 𝑃(𝑗)
|𝑂𝑋𝑖𝑗ሬሬሬሬሬሬሬሬԦ| = 𝑃(𝑖,𝑗)
Item i Item jItem i 1 P(j|i)Item j P(i|j) 1
X ｊ
Xi
O
Xij
𝑂𝑋𝑖𝑗ሬሬሬሬሬሬሬሬԦ= −𝑋𝑖𝑋𝑗ሬሬሬሬሬሬሬሬԦ∙𝑂𝑋𝑖ሬሬሬሬሬሬሬԦ|𝑋𝑖𝑋𝑗ሬሬሬሬሬሬሬሬԦ| 𝑋𝑖𝑋𝑗ሬሬሬሬሬሬሬሬԦ+ 𝑂𝑋𝑖ሬሬሬሬሬሬሬԦ
Relationship betweenitems i and j
|𝑂𝑋𝑖ሬሬሬሬሬሬሬԦ| = 𝑃(𝑖) |𝑂𝑋𝑗ሬሬሬሬሬሬሬԦ| = 𝑃(𝑗)
|𝑂𝑋𝑖𝑗ሬሬሬሬሬሬሬሬԦ| = 𝑃(𝑖,𝑗)
Relationship betweenitems i and j
Item i Item jItem i 1 P(j|i)Item j P(i|j) 1
X ｊ
Xi
O
Xij
𝑃ሺ𝑖ȁ�𝑗ሻ= 𝑃(𝑖,𝑗)𝑃(𝑗)
𝑃ሺ𝑗ȁ�𝑖ሻ= 𝑃(𝑖,𝑗)𝑃(𝑖)
|𝑂𝑋𝑖𝑗ሬሬሬሬሬሬሬሬԦ||𝑂𝑋𝑖ሬሬሬሬሬሬሬԦ|
|𝑂𝑋𝑖𝑗ሬሬሬሬሬሬሬሬԦ||𝑂𝑋𝑗ሬሬሬሬሬሬሬԦ|
𝑐𝑜𝑠∠𝑋𝑖𝑂𝑋𝑖𝑗
𝑐𝑜𝑠∠𝑋𝑗𝑂𝑋𝑖𝑗
Asymmetric correct response rate matrix
• The asymmetric matrix lacks information about the correct response rate of each item
• So we add the imaginary n+1-th item whose correct response rate is 1.0
– P(j|n+1)=P(j,n+1)/P(n+1)=P(j)– P(n+1|j)=P(j,n+1)/P(j)=1.0
Item 1 Item 2 ⋯ Item nItem 1 1 P(2|1) ⋯ P(n|1)Item 2 P(1|2) 1 ⋯ P(n|2)⋮ ⋮ ⋮ ⋱ ⋮
Item n P(1|n) P(2|n) ⋯ 1
Item 1 Item 2 ⋯ Item n Item n+1Item 1 1 P(2|1) ⋯ P(n|1) P(n+1|1)Item 2 P(1|2) 1 ⋯ P(n|2) P(n+1|2)⋮ ⋮ ⋮ ⋱ ⋮ ⋮
Item n P(1|n) P(2|n) ⋯ 1 P(n+1|n)Item n+1 P(1|n+1) P(2|n+1) ⋯ P(n|n+1) 1
Item 1 Item 2 ⋯ Item n Item n+1Item 1 1 P(2|1) ⋯ P(n|1) 1Item 2 P(1|2) 1 ⋯ P(n|2) 1⋮ ⋮ ⋮ ⋱ ⋮ ⋮
Item n P(1|n) P(2|n) ⋯ 1 1Item n+1 P(1) P(2) ⋯ P(n) 1
Expanded
Stress function
otherwisen
njiifij 1
,1
Not
formedWellij 0
1
)1(
)|(1
)(,
nn
jipp
n
jiji
22
22
||||
)'(||||
||
||)|(
jij
ijij
j
ij
OX
OXji
xxx
xxxx
1
)(,
2
1
)(,
2
*
)|(
)|()|(
)(
)()( n
jiji ijij
n
jiji ij
pjiλδ
jijipλ
T
FF
X
XX
Xij
δ (delta)Well-formed triangle
• The perpendicular foot from O falls on line segment XiXj
Not well-formed triangle
• The foot from O does NOT fall on line segment XiXj
Xi
X ｊ
O
Xij
XiX ｊ
O
δij=δji=1
δij=δji=0
λ （ lambda ）
Item 1 Item 2 ⋯ Item n Item n+1Item 1 1 P(2|1) ⋯ P(n|1) 1Item 2 P(1|2) 1 ⋯ P(n|2) 1⋮ ⋮ ⋮ ⋱ ⋮ ⋮
Item n P(1|n) P(2|n) ⋯ 1 1Item n+1 P(1) P(2) ⋯ P(n) 1
0.5
0.51
1
otherwisen
njiifij 1
,1
Spatial indeterminacy and fixed coordinates
• Number of dimensions=3
• Coordinates of item n+1– (xn+1=0, yn+1=0, zn+1=1)
• Coordinate of item k, which has the lowest correct response rate– (xk=0, yk>0, zk)
• Coordinate of item l, which has a moderate P( ・ |k)– (xl>0, yl, zl)
Demonstration of exametrika
13
www.rd.dnc.ac.jp/~shojima/exmk/index.htm
Result of Analysis: Radial Map
• Red dots– Estimated coordinates
• Orange dots – Points of intersections
of extensions of red line segments and the surface of the hemisphere
Relationship betweenimaginary item n+1and item j
• P(j)→1.0• P(k)→0.0
Item j Item n+1Item j 1 1
Item n+1 P(j) 1
Xj
P(j)1
P(k)
Xk
Xn+1
O
Relashinship between items i and j
• P(i)<P(j)• P(i|j)→1.0• P(i|j)→0.0
Item i Item jItem i 1 P(j|i)Item j P(i|j) 1
Xn+1
Xi
O
Xj
Topographic Map
• The coordinates of orange points are projected onto the XY plane
• Voronoi tessellation
• Lift each Voronoi region by the length of the orange line segment
– Separate height with different colors
Mastery Maps
• For each examinee
Demonstration of exametrika
19
www.rd.dnc.ac.jp/~shojima/exmk/index.htm
Thank you for listening.
SHOJIMA KojiroThe National Center for University Entrance Examinations