The Polytomous Unidimensional Rasch Model: Understanding its Response Structure and Process

ACSPRI Social Science Methodology Conference, Sydney, December 2006

Mailing addressDavid AndrichMurdoch University Murdoch 6150Western AustraliaEmail: andrich@murdoch.edu.au

Acknowledgements

• The research was supported in part by an Australian Research Council Linkage grant with the Australian National Ministerial Council on Employment, Education, Training and Youth Affairs (MCEETYA) Performance Measurement and Reporting Task Force; UNESCO’s International Institute for Educational Planning (IIEP), and the Australian Council for Educational Research (ACER) as Industry Partners.

Ingredient 1Invariance of comparisons

• Rasch’s requirement for invariance of parameters estimates leads to models with sufficient statistics

• The model for dichotomous responses is a special case

Ingredient 2Standard formats

Fail < Pass < Credit < Distinction

Never < Sometimes < Often < Always

Strongly Disagree < Disagree < Agree < Strongly Agree

Ingredient 3The dichotomous Rasch model

• Criterion

• The dichotomous model

),,,()},(|,,);,{( jinjninjnijinnjnjnini yyyyfyYyYr

in

in

eeyY

y

ni

1}Pr{

)(

The Item Characteristic Curvesand thresholds i

Ingredient 4The Guttman Structure

Items 1 2 3 Total

Score x }|),,Pr{( 321 xyyy nnn

I+1=4 Guttman response patterns 0 0 0 0 1 1 0 0 1 0.667 1 1 0 2 0.678 1 1 1 3 1

2I– I–1 =4 Non-Guttman response patterns 0 1 0 1 0.248 0 0 1 1 0.085 1 0 1 2 0.235 0 1 1 2 0.087

Ingredient 5Design of an experiment

_____________________________________________________________________ Fail (F) Inadequate setting Insufficient or irrelevant information given for

the story… Pass (P) Discrete setting Discrete setting as an introduction, with some

details which also show some linkage and organisation. …. Credit (C) Integrated setting There is a setting which, rather than simply

being at the beginning, is introduced throughout the story. Distinction (D) Integrated and manipulated setting: In addition to the setting

being introduced throughout the story, … _____________________________________________________________________

Experimentally independent responses

Inadequate setting F

Discrete setting P

Integrated setting C

Integrated and manipulated

setting D

Judge 1 Not P P

Judge 2 Not C C

Judge 3 Not D D

Requirements of data

• 1. The success rate at P is greater than that at C, and that the success rate at C is turn be greater than at D

• 2. Relative success rates are independent of the locations of the essays on the continuum. (The dichotomous Rasch model)

DCP ˆˆˆ

Ingredient 6Analysis of Sample Spaces

}1,0{},{ yyYnix

imx ...2,1

nixnixnixnix QPPY 1;}1Pr{

The complete response space

patterns

)},...,...,{( 21 inimnixnini yyyy

im2

iny 1 iny 2 iny 3

0

1niQ 0

2niQ 0

3niQ

1

1niP 0

2niQ 0

3niQ

1

1niP 1

2niP 0

3niQ

' 1

1niP 1

2niP 1

3niP

0

1niQ 1

2niP 0

3niQ

0

1niQ 0

2niQ 1

3niP

1

1niP 0

2niQ 1

3niP

0

1niQ 1

2niP 1

3niP

1)},...,...,Pr{( 3

1

121

k

m

k

ynki

ynkinminkiinin

inkinki QPyyyy

The Guttman subspace

iny 1 iny 2 iny 3

0

1niQ 0

2niQ 0

3niQ

1

1niP 0

2niQ 0

3niQ

1

1niP 1

2niP 0

3niQ

' 1

1niP 1

2niP 1

3niP

1)},...,...,Pr{(''

3

1

121

k

m

k

ynki

ynkinminkiinin

inkinki QPyyyy

Define

},...,2,1,0{;1

i

m

knikni mxyxX

i

Inferred category and the

total score xX ni

1niy

2niy 3niy

0 Fail 0 0 0 1 Pass 1 0 0 2 Credit 1 1 0 ' 3 Distinction 1 1 1

A sub subspace

Inferred category and the

total score xX ni

1niy

2niy 3niy

0 Fail 0 0 0 1 Pass 1 0 0 1,0

' 2 Credit 1 1 0 ' 3 Distinction 1 1 1

Probability of a response in the Guttman Space

DQPyyyyk

m

k

ynki

ynkinminkiinin

inkinki

1)},...,...,Pr{(''

3

1

121

iiii nimninininimninininimninininimninini PPPPQQPPQQQPQQQQD ............... 321321321321

DQQQPPPPxXinimnixnixnixnininini /.......}|Pr{ 21321

'

1}|Pr{0

'

im

xni xX

The doubly conditioned outcome space xx ,1

'

nixnixnix

nix

nini

ni PQP

PxXxX

xX

}|Pr{}|1Pr{}|Pr{

''

'

}|Pr{}|1Pr{}|Pr{

}|Pr{ ''

''

,1

xXxXxX

xXPnini

nixxninix

nixnixxxni PYxX }|1Pr{}|Pr{ ',1

Reverse Process

Let

Define

Then

Indicating the model is the same

},...,2,1,0{, ini mxxX

nixnini

ni PxXxX

xX

}Pr{}1Pr{}Pr{

nixnix PQ 1

DQQQPPPPxXinimnixnixnixnininini /.......}|Pr{ 21321

'

Inferring an experimentally independent outcome space

• Given the Guttman space , we infer the existence of a complete space of which is a subspace. In this complete space we can infer experimentally independent responses.

' '

Construction and interpretation of the PRM

Let ixn

ixn

eePnix

1

ixnixn

ixn

eeeQnix

11

11

The PRM

Deee

ee

ee

e

xX

iimnixnixn

ixn

in

in

in

in

ni

/1

1...1

11

...11

}|Pr{

12

2

1

1

'

i

x

kiknm

x

x

ni e0

0

ni

x

ni

x

kikn

exX

/}|Pr{ 0'

Equivalences of corresponding thresholds in the spaces

Outcome space Response Px Cx Dx

}|1Pr{ nixY

iPn

iPn

ee

1

iCn

iCn

ee

1

iDn

iDn

ee

1

}|Pr{ ',1 xxni xX

iFPn

iFPn

ee

1

iPCn

iPCn

ee

1

iCDn

iCDn

ee

1

iFPiP iPCiC iCDiD

{ }|{}| ',1 xixix

Simulation 1 Judges Item

number Generated locations

Estimates from the dichotomous RM in a full space

Estimates from the PRM in a Guttman space

Novice 1 (P) 1 -1.1 -1.125 -1.047 Novice 2 (C) 2 -0.1 -0.119 -0.133 Novice 3(D) 3 1.2 1.267 1.196 Expert 1(P) 4 -1.7 -1.685 -1.680 Expert 2 (C) 5 0.3 0.227 0.232 Expert 3 (D) 6 1.4 1.435 1.433 RMSQ 0.018 0.017 Fit 2 =6.081, df=4

P <0.193

Category and latent dichotomous probabilities

Novice Expert

Simulation 2Judges Item

number Generated locations

Estimates from the dichotomous RM in a full space

Estimates from the PRM in a Guttman space

Novice 1 (P) 1 -1.1 -1.125 -1.132 Novice 3 (C) 2 1.2 1.267 1.452 Novice 2(D) 3 -0.1 -0.119 -0.169 Expert 2(P) 4 0.3 0.227 0.122 Expert 1 (C) 5 -1.7 -1.685 -1.651 Expert 3 (D) 6 1.4 1.435 1.379 Extreme scores eliminated

1214 1214

RMSQ 0.018 0.054 Fit 2 =0.932, df=4

P <0.920

Category and latent dichotomous probability curves

Novice Expert

Item S121 Marking Key 2: all parts recognisable in shape, size, position, and orientation. 1: most parts recognisable in shape, size, position, and orientation. 0: few parts recognizable in shape, size, position, and orientation.

Item S004 Marking Key 2:rectangle well drawn 1:correct daisies chosen, but rectangle poorly drawn 0:not a four sided figure

Category and latent dichotomous probability curves

Mathematics Item S121 Mathematics Item S004