1

An ordinal IRT model for a circular representation

of polytomous data

Wijbrandt H. van SchuurUniversity of Groningen

25th Workshop on Item Response Theory University of TwenteOctober 12-15, 2009

2

Overview

A. From dominance model to proximity model

and from monotone to circular proximity

B. From dichotomous to polytomous data

(C. From deterministic to probabilistic model)

3

IRF’s of three dominance items

Figure 7: Three doubly monotone items

0

0,5

1

-8 -6 -4 -2 0 2 4 6 8

latent continuum

pro

bab

ilit

y p

osi

tive

res

po

nse

4

Example dominance model (World Values Study)

Subjects A: Hell B: Devil C: Heaven D: God

1 0 0 0 0

2 1 0 0 0

3 1 1 0 0

4 1 1 1 0

5 1 1 1 1

Do you believe in …Item A HellItem B The DevilItem C HeavenItem D God

5

IRF’s of six monotone proximity items

CN

9,15

8,07

7,18

6,27

5,21

4,41

3,39

2,59

1,74

,62

-,46

-1,72

-2,87

-4,04

-5,18

-6,09

-7,08

-7,91

-9,11

-9,98

Me

an1,2

1,0

,8

,6

,4

,2

0,0

P40

P41

P42

P43

P44

P45

6

Example Monotone proximity model(Electoral compass)

item Clinton Obama Edwards Richard-son

McCain Hucka-bee

Romney Thomson Giuliani

1 1 1 0 0 0 0 0 0 0

2 1 1 1 1 1 0 0 0 0

3 0 0 0 1 1 0 0 0 0

4 0 0 0 1 1 1 1 1 0

5 0 0 0 0 1 1 1 1 0

6 0 0 0 0 0 0 1 1 1

Item 1 The best way to reduce the federal deficit is to increase taxes Item 2 Mortgage lenders should be more tightly controlled Item 3 The US should decrease its spending on defense Item 4 Stricter gun control will not reduce crime Item 5 Abortion should be made completely illegal Item 6 The US should never sign international treaties on climate change

that limit economic growth

7

7,006,005,004,003,002,001,000,00

1,00

0,80

0,60

0,40

0,20

0,00

Vertical: probability of positive response

Horizontal: items i, j and k scale values (in radians between 0 and 2π)

IRF’s of three circular proximity items

0o 60o 120o 180o 240o 300o 360o = 0o

8Larsen, R.J. & Diener (1992), E. Promises and problems with the circumplex model of emotion, p. 31. In: M.S. Clark & J.R. Averill (eds.). Emotion: Review of personality and social psychology (Vol. 13, pp. 25-59), Newbury Park, CA: Sage.

Larsen & Diener

9

Brown, M.W. (1992). Circumplex models for correlation matrices. Psychometrika, 57, 470, 479

Brown: Vocational Interests

R

I

A

S

E

C

10

Schwartz: Universals in the content and structure of values

Stability

Other Self

Change

Security

Conformity

Benevolence

Universalism

Self Direction

Stimulation

Hedonism

Achievement

Power

Schwartz, S.H. (1992). Universals in the content and structure of values: theoretical advances and empirical tests in 20 countries.

In: M.P. Zanna (ed.), Advances in experimental social psychology, Vol. 25 (p. 1-65). San Diego/London: Academic Press..

11

Example circular proximity model (First two dimensions of Big FIVE)

Active (N)

Lively (NE)

Glad (E)

Calm (SE)

Still (S)

Tired (SW)

Sad (W)

Anxious (NW)

1 1 1 1 0 0 0 0 0

2 0 1 1 1 0 0 0 0

3 0 0 1 1 1 0 0 0

4 0 0 0 1 1 1 0 0

5 0 0 0 0 1 1 1 0

6 0 0 0 0 0 1 1 1

7 1 0 0 0 0 0 1 1

8 1 1 0 0 0 0 0 1

12

Violations of deterministic models

Dominance model: 1 subject and 2 items: 01-response to item pair (Mokken, 1971)

Monotone proximity model: 1 subject and 3 items: 101-response to item triple (Van Schuur, 1984)

Circular proximity model: 1 subject and 4 items: 1010- or 0101 response

to item quadruple (Leeferink, 1997, Mokken, van Schuur & Leeferink, 2001)

13

Homogeneity (Loevinger)For each elementary scale (pair, triple, quadruple): H = 1 - E(obs)/E(exp) = φ/φmax E(exp): product of relevant probabilities * N (for dominance data)

For each item: Hi = 1 – Σ E(obs)/Σ E(exp) Summation over all elementary scales that contain item i

For whole scale: H = 1 – Σ E(obs)/Σ E(exp) Summation over all elementary scales

Person fit: number of elementary scales in response pattern that contain a model violation

14

ExplorationBottom-up hierarchical clustering procedure:

1. “Best” elementary scale

2. “Best” next item

15

Item steps and subject scale valuesDominance model: 2 items i and j, 2 categories i: 0 1 1 j: 0 0 1 sum: 0 1 2 ──────┬─────┴────┬───┴───┬──────

θ0 δi01 θ1 δj01 θ2

Proximity models: 3 items i,j,k 2 categories i: 0 1 1 1 0 0 0 j: 0 0 1 1 1 0 0 k: 0 0 0 1 1 1 0 sum: 0 1 2 3 4 5 6

───┬─┴─┬─┴──┬─┴─┬─┴─┬─┴─┬┴─┬ δi01 δj01 δk01 δi10 δj10 δk10

16

Item steps and subject scale valuesDominance model: 2 items i and j, 2 categories i: 0 1 1 j: 0 0 1 sum: 0 1 2 ──────┬─────┴────┬───┴───┬──────

θ0 δi01 θ1 δj01 θ2

Proximity models: 3 items i,j,k 2 categories i: 0 1 1 1 2 2 2 j: 0 0 1 1 1 2 2 k: 0 0 0 1 1 1 2 sum: 0 1 2 3 4 5 6

───┬─┴─┬─┴──┬─┴─┬─┴─┬─┴─┬┴─┬ δi01 δj01 δk01 δi10 δj10 δk10

17

I01

I10

J01

J10

K10

K01

L01

L10

1100=2

1110=3

0110=4

0111=50011=6

0001=7

1001=0 or 8

1000=1 I

J

K

L

Scale values of subjects

18

Scale value of items: dominance model

ORDER of the items is generally based on popularity in sample

19

Scale value of items: unfolding model

ORDER of the items steps is based on uniqueness of representation (popularity is irrelevant)

Which item is middle item: BAC, ABC, or ACB? Requirement for “best” triple: “unique” triple: Positive homogeneity in only one permutation

20

Scale value of items: circumplex model

For circular proximity model: a. Each item can be the first in an ordered quadruple: ABCD = BCDA = CDAB = DABC b. Clockwise and counter clockwise: ABCD=DCBA

So, arbitrarily beginning with item A: Which item is middle item among remaining three items : CBD, BCD, or BDC? (or quadruples ACBD, ABCD, or ABDC) Requirement for “best” quadruple: “unique” quadruple: Positive homogeneity in only one permutation

21

Polytomous itemsDominance Monotone Circular

Model Proximity Model Proximity ModelCumulative scale Unfolding scale Circumplex scaleA B C D E F A B C D E F G A B C D E F G 0 0 0 0 0 0 2 1 1 0 0 0 0 1 2 1 0 0 0 01 0 0 0 0 0 1 2 1 0 0 0 0 0 1 2 1 0 0 01 1 0 0 0 0 0 1 2 1 0 0 0 0 0 1 2 1 0 02 2 1 0 0 0 0 0 1 2 1 0 0 0 0 0 1 2 1 02 2 1 1 0 0 0 0 0 1 2 1 0 0 0 0 0 1 2 12 2 2 2 1 1 0 0 0 0 1 2 1 1 0 0 0 0 1 22 2 2 2 2 2 0 0 0 0 1 1 2 2 1 0 0 0 0 1

For dominance model: Molenaar 1983For unfolding model: Van Schuur 1993

22

Item steps of polytomous itemsDominance model: 2 items i and j, 4 categories i: 0 1 1 2 3 3 3 j: 0 0 1 1 1 2 3 ──┴────┴─────┴────┴───┴──┴───

δi01 δj01 δi12 δi23 δj12 δj23

Proximity models: 1 item i, 4 categories i: 0 1 2 3 2 1 0

──┴────┴─────┴────┴───┴──┴─── δi01 δi12 δi23 δi32 δi21 δi10

23

Model violations for polytomous dominance data

Dominance model: 1 subject and 2 item steps

Weight of seriousness of model violation: i: 0 1 1 2 3 3 3 j: 0 0 1 1 1 2 3 ──┴────┴─────┴────┴───┴──┴───

δi01 δj01 δi12 δi23 δj12 δj23

(i=0,j=1) is less serious than (i=0, j=3)

24

Model violations for polytomous unfolding data

Monotone Proximity model: Response pattern ABC,323 less bad than ABC,302

Concept of ‘implicit error”: Given ABC,302: AB=30, so C must be 0 and C=1 and C=2 are errors. C=1 is an implicit error, C=2 is the explicit error

AC=32, so B must be 2 or 3, and B=1, B=0 in error B=1: implicit; B=0: explicit

BC=02, so A must be 0, and A=1, A=2, A=3: error A=1 and A=2: implicit; A=3: explicit

Weight of errors in triple: sum of implicit and explicit errors in pairs of triple.

25

Model violations for polytomous circumplex data

Circular Proximity model: Response pattern ABCD,3232 less bad than ABCD,3021

Concept of ‘implicit error”: Given ABCD,3021: ABC=302, so D must be 2 or 3; D=1 is the explicit error

ABD=301, so C must be 0 or 1, and C=2 explicit error

ACD=321, so B must be 2 or 3, and B=1 or B=0 are errors B=0 is explicit error and B=1 is implicit error

BCD=021, so A must be 0 or 1, and A=2 or A=3 are errors A=3: explicit and A=2: implicit

Weight of errors in quadruple: sum of implicit and explicit errors in triples of quadruple.

26

Homogeneity for polytomous circumplex data

For elementary scale: H = 1 – Σ W* E(obs) / Σ W*E(exp) Summation over relevant elementary item step combinations

For item or whole scale: H = 1 – ΣΣ W*E(obs) / ΣΣ W*E(exp) + Summation over relevant triples

27

J 1

L 1

K 1

K 1

K 2

L 2

K

L

k(21)

k(12)

l(12)j(10)

l(01)

j(21)

i(10)

I

J

k(01)

I 1

I 2

I 1

J 2

J 1i(12)

i(21)

i(01)

j(12)

L 1

j(01)

k(10)

l(21)

l(10)

1001=0 or 16

0110=8

2001=1

2101=22100=3

2200=4

1200=5

1210=6

0210=7

0120=9

0121=10

0021=11

0022=12

0012=13

1012=14

1011=15

28

Problems with scale values for subjects200000 unfalsifiable: no model error possible

202020 symmetrical: unscalable

101100 no highest value (2): ambiguous (change to 202200)

Imperfect patterns: calculation clockwise and counterclockwise should give the same result. If not: response pattern is symmetrical (unscalable) or take mean of both values

29

Probabilistic model: Use diagnostic matrices

Shape of Correlation matrix (high-low-high values)

Similarly: shape of (Conditional) Adjacency matrix shape of Dominance matrix

In development: criteria analogous to criteria developed for the Mokken scale by Molenaar and Sijtsma

30

What can we do with circular subject scores?

Biologists: compass and clock Mardia (1972): Statistics of directional data Batschelet (1981): Circular Statistics in biology Fisher (1993): Statistical analysis of circular data

Compare distributions (uniform, unimodal) Use circular scale values as dependent or independent variable in regression analyses

31

THANK YOU

32

Correlation Matrix: values decrease from the diagonal towards the lowest value (underlined), and then increase again towards the diagonal.

A B C D E F G H I A 1.00 0.50 0.07 -0.14 -0.32 -0.29 -0.18 0.07 0.54 B 0.50 1.00 0.57 0.07 -0.18 -0.36 -0.32 -0.14 0.11 C 0.07 0.57 1.00 0.50 0.11 -0.21 -0.32 -0.29 -0.18 D -0.14 0.07 0.50 1.00 0.54 0.14 -0.18 -0.36 -0.32 E -0.32 -0.18 0.11 0.54 1.00 0.54 0.07 -0.18 -0.29 F -0.29 -0.36 -0.21 0.14 0.54 1.00 0.54 0.07 -0.18 G -0.18 -0.32 -0.32 -0.18 0.07 0.54 1.00 0.54 0.07 H 0.07 -0.14 -0.29 -0.36 -0.18 0.07 0.54 1.00 0.54 I 0.54 0.11 -0.18 -0.32 -0.29 -0.18 0.07 0.54 1.00

33

Conditional Adjacency Matrix:Response value (>=): 1

A B C D E F G H I A 1.00 0.72 0.52 0.45 0.34 0.38 0.41 0.52 0.76 B 0.78 1.00 0.78 0.56 0.41 0.33 0.33 0.41 0.56 C 0.56 0.78 1.00 0.78 0.56 0.41 0.33 0.33 0.41 D 0.45 0.52 0.72 1.00 0.76 0.59 0.41 0.31 0.34 E 0.36 0.39 0.54 0.79 1.00 0.79 0.54 0.39 0.36 F 0.38 0.31 0.38 0.59 0.76 1.00 0.76 0.52 0.41 G 0.43 0.32 0.32 0.43 0.54 0.79 1.00 0.75 0.54 H 0.56 0.41 0.33 0.33 0.41 0.56 0.78 1.00 0.78 I 0.79 0.54 0.39 0.36 0.36 0.43 0.54 0.75 1.00

34

Dominance Matrix:Response value (>=): 1

A B C D E F G H I A 0.00 0.14 0.25 0.29 0.34 0.32 0.30 0.25 0.13 B 0.11 0.00 0.11 0.21 0.29 0.32 0.32 0.29 0.21 C 0.21 0.11 0.00 0.11 0.21 0.29 0.32 0.32 0.29 D 0.29 0.25 0.14 0.00 0.13 0.21 0.30 0.36 0.34 E 0.32 0.30 0.23 0.11 0.00 0.11 0.23 0.30 0.32 F 0.32 0.36 0.32 0.21 0.13 0.00 0.13 0.25 0.30 G 0.29 0.34 0.34 0.29 0.23 0.11 0.00 0.13 0.23 H 0.21 0.29 0.32 0.32 0.29 0.21 0.11 0.00 0.11 I 0.11 0.23 0.30 0.32 0.32 0.29 0.23 0.13 0.00

35

Score group matrix. Response value (>=): 1score N scale A B C D E F G H I group range 1 6 (17- 0) 1.00 0.67 0.33 0.00 0.00 0.17 0.50 0.83 1.00 2 6 ( 1- 2) 1.00 1.00 0.67 0.33 0.00 0.00 0.17 0.50 0.83 3 6 ( 3- 4) 0.83 1.00 1.00 0.67 0.33 0.00 0.00 0.17 0.50 4 6 ( 5- 6) 0.50 0.83 1.00 1.00 0.67 0.33 0.00 0.00 0.17 5 10 ( 7- 9) 0.20 0.40 0.70 1.00 1.00 0.80 0.40 0.10 0.10 6 10 (10-12) 0.10 0.00 0.20 0.60 0.80 1.00 0.90 0.50 0.20 7 6 (13-14) 0.33 0.00 0.00 0.17 0.50 0.83 1.00 1.00 0.67 8 6 (15-16) 0.67 0.33 0.00 0.00 0.17 0.50 0.83 1.00 1.00

36

ExplorationBottom-up hierarchical clustering procedure: 1. “Best” elementary scale 2. “Best” next item

Ad 1: - High(est) homogeneity - highest number of subjects who use items of elementary scale in acceptable pattern (for proximity models) - unique representation (for proximity models)

Ad 2: - High(est) homogeneity - unique representation (for proximity models)