scale construction
TRANSCRIPT
-
7/28/2019 Scale Construction
1/111
Scale Construction
et o s, tat st cs, an o e s
Fridtjof Nussbeck, University of Zurich
-
7/28/2019 Scale Construction
2/111
Outline
psychological assessment
criteria for valid assessment
scale development
ISSAS 2009 - scale construction 2
p ases o sca e eve opment statistics
MTMM models
outlook
2ISSAS 2009 - scale construction
-
7/28/2019 Scale Construction
3/111
Psychological Measurement
systematic measurement of a persons behavior
different strategies to assess target persons
inferences (and clinical judgments) drawn
ISSAS 2009 - scale construction 3
rom t e resu ts
3ISSAS 2009 - scale construction
-
7/28/2019 Scale Construction
4/111
I. Assessment Methods direct observation
psychophysiological measurement questionnaires
ISSAS 2009 - scale construction 4
-emotional states / mood / well being
single-shot studies
repeated measures
diary methods
ambulatory assessment
4ISSAS 2009 - scale construction
-
7/28/2019 Scale Construction
5/111
Measurement and ValidityAll decisions (assessments) in psychology should be basedon the best information available. Information is best when
it is objective, reliable, valid, and specific to a givenproblem.
Reliable Measures:
ISSAS 2009 - scale construction 5
measurement error is small Cronbachs alphaValid Measures:no impact on measurement scores than those one wants to
measure Fit into nomological net
Burns & Haynes (2006); Messick (1995); Courvoisier, Nussbeck, Eid, Geiser, & Cole (2008)
55
-
7/28/2019 Scale Construction
6/111
II. Scale Construction
not just:
looking for some words that tap the construct
ISSAS 2009 - scale construction 6
Standards forEducational and Psychological Testing
(American Psychological Association)
6ISSAS 2009 - scale construction
-
7/28/2019 Scale Construction
7/111
When to Construct a New Scale? no scale exists measuring a specific construct
existing scales do not represent the constructadequately:
lack of reliabilit
ISSAS 2009 - scale construction 7
lack of validity
outdated (old words; meaning of words changed;attitudes changed)
trait measure vs. state measure
insensitive for changes
7ISSAS 2009 - scale construction
-
7/28/2019 Scale Construction
8/111
Phases of test / scale construction
ISSAS 2009 - scale construction 88ISSAS 2009 - scale construction
-
7/28/2019 Scale Construction
9/111
Construct Definitionscope of the scale (level of abstraction)
what is to be measured?
broad or narrow construct?
ISSAS 2009 - scale construction 9
we - e ng vs. spec c emo on
definition of the construct in literature?
adopt a definition vs. work out an own definition
clearly describe what is meant by your construct(and what is not meant)
9ISSAS 2009 - scale construction
-
7/28/2019 Scale Construction
10/111
Construct Definitionaspects of construct
continuum vs. categories
frequency / intensity of experiencing emotions
ISSAS 2009 - scale construction 10
,
scores on items can be combined using (weighted)
means
emotional reactions (baroque, rational, positive
reappraising)
response patterns can be analyzed yielding classes ofreaction styles
1010
-
7/28/2019 Scale Construction
11/111
Construct Definition unidimensional vs. multidimensional
frequency of experiencing specific positive emotions
love, affection, intimacy, security
joy, happiness, cheerfulness, contentment
fre uenc of ex eriencin ositive affect
ISSAS 2009 - scale construction 11
Love, affection, joy, happiness
facets of a construct
positive affect may comprise
love Happiness
pride
11ISSAS 2009 - scale construction
-
7/28/2019 Scale Construction
12/111
Recommendation write out a brief, formal description of the
construct relate it to other constructs
ISSAS 2009 - scale construction 12
dimensionality / facetsmay also help to avoid known problems with
respect to unclear instructions, problematic
response formats, etc.
12ISSAS 2009 - scale construction
-
7/28/2019 Scale Construction
13/111
-
7/28/2019 Scale Construction
14/111
Design Scalethe initial item pool
is broader than ones own theoretical view about theconstruct
includes multiple items for each potential facet /dimension
ISSAS 2009 - scale construction 14
includes also items that will finally proof to be distinct ortangential to the construct
search for aspects in literature (scientific, but also fiction,dictionnaries, etc.)
ask friends and family
item generation is a creative act
14ISSAS 2009 - scale construction
-
7/28/2019 Scale Construction
15/111
Design Scalebasic principles of item writing
simple, straightforward, and appropriatelanguage
ISSAS 2009 - scale construction 15
a equa e o rea ng eveno trendy expressions / colloquialisms
Ex.:
Do you feel happy? never sometimes always
15ISSAS 2009 - scale construction
-
7/28/2019 Scale Construction
16/111
Design Scalebasic principles of item writing
one aspect at a time
I feel happy and beloved
ISSAS 2009 - scale construction 16
I o not insu t peop e ecause it is mora y wrong individuals must differ on items
constants are useless
I am the happiest individual alive
16ISSAS 2009 - scale construction
-
7/28/2019 Scale Construction
17/111
Design Scalebasic principles of item writing
avoid frequencies in item wordingSometimes, I am happy
ISSAS 2009 - scale construction 17
terms
I worry about neuroticism
avoid negatives to reverse meaning of an item,but include negative aspectsI am not happy I am sad
17ISSAS 2009 - scale construction
-
7/28/2019 Scale Construction
18/111
Multidimensional Mood Questionnaire
ISSAS 2009 - scale construction 18
tems o t e easant-Unpleasant dimension
Steyer, Schwenkmezger, Notz, & Eid (1994)
-
7/28/2019 Scale Construction
19/111
Semantic Opposites?
vs.
ISSAS 2009 - scale construction 19
-
7/28/2019 Scale Construction
20/111
Semantic Opposites?
ISSAS 2009 - scale construction 20
-
7/28/2019 Scale Construction
21/111
Design Scaleresponse format
depends on introductionHow often do you feel .
frequency format (never, always)
ISSAS 2009 - scale construction 21
Do you agree to. Agreement (not at all, very much so)
How much does correspond to you
similarity (not at all like me, very much like me)
How do you judge
evaluation (terrible, excellent)
21ISSAS 2009 - scale construction
-
7/28/2019 Scale Construction
22/111
Design Scaleresponse format
avoid middle category to increase variability
not always desirable
avoid too many categories, respondents cannot
ISSAS 2009 - scale construction 22
eren a e a equa e y ana og sca e
avoid check-lists response bias
avoid forced choice formats
indicates relative strength of alternatives and cannot becompared across individuals (no normative interindividual
information)
22ISSAS 2009 - scale construction
-
7/28/2019 Scale Construction
23/111
Design Scale standard metrical analysis techniques work well with
five or more categories However, labels from 1 to 5 do not guarantee a
metric scale!
ISSAS 2009 - scale construction 23
-
7/28/2019 Scale Construction
24/111
Pilot Test small sample
critique scaleambiguous or confusing items
ISSAS 2009 - scale construction 24
mismatc o items an sca e
24ISSAS 2009 - scale construction
-
7/28/2019 Scale Construction
25/111
Administration and Item Analysislarge set of items (all potential items)
considerable sample size (about 100)
correlation with criteria (nontest-criteria)
ISSAS 2009 - scale construction 25
internal consistency (interitem structure) item response theory (focused on latent trait)
25ISSAS 2009 - scale construction
-
7/28/2019 Scale Construction
26/111
Validate and NormValidity:
does the scale measure, what it intends tomeasure?
content validit
ISSAS 2009 - scale construction 26
convergent validity
criterion validity
discriminant validity
Norms:
what are the properties of the distribution of scores
for a given population?26ISSAS 2009 - scale construction
-
7/28/2019 Scale Construction
27/111
StatisticsBefore you start:
items recoded?save recoded items using a new name
ISSAS 2009 - scale construction 27
look at distributions!
ISSAS 2009 - scale construction 27Wut
543210
Frequency
300
200
100
0
Wut
Mean =1.86
Std. Dev. =0.71
N =481
1
2
3
4
Groll
AAAAAA
-
7/28/2019 Scale Construction
28/111
Statisticsfirst simple analysis:
item correlations with criteriondichotomous with interval / dichotomous
ISSAS 2009 - scale construction 28
interval with interval Pearson
dichotomous / ordinal with ordinalWilcoxon
ISSAS 2009 - scale construction 28
-
7/28/2019 Scale Construction
29/111
Internal Structureinteritem correlations
factor analysis for metrical outcomesstandard factor analysis:
ISSAS 2009 - scale construction 29
exp ora ory ac or ana ys s
confirmatory factor analysis
factor analysis for ordinal data
special estimators (WLS / WLSMV) required
ISSAS 2009 - scale construction 29
-
7/28/2019 Scale Construction
30/111
Interitem Correlations
ISSAS 2009 - scale construction 30
-
7/28/2019 Scale Construction
31/111
Exploratory Factor Analysishow many underlying latent variables exist in
the current set of items?
ISSAS 2009 - scale construction 31ISSAS 2009 - scale construction 31
i i i i
or
j 2Y j j j = + +
i 1Y i i i = + + for some items i
for some (other) items j
-
7/28/2019 Scale Construction
32/111
Exploratory Factor Analysisdata driven approach to reduce complexity
model finding
depends strongly on input (items)
ISSAS 2009 - scale construction 32
you get out, what you put in
-
7/28/2019 Scale Construction
33/111
Criteria for Factor AnalysisHow many factors are needed?
scree plot eigenvalues
parallel analysis
ISSAS 2009 - scale construction 33
explained variance (goodness of fit coefficients)
number of factors does not depend on extraction
methodfor ease of interpretation choose (varimax) rotation
-
7/28/2019 Scale Construction
34/111
Scree Plot
ISSAS 2009 - scale construction 34
-
7/28/2019 Scale Construction
35/111
Eigenvalues & Explained Variance
ISSAS 2009 - scale construction 35
-
7/28/2019 Scale Construction
36/111
Factor Loadings
ISSAS 2009 - scale construction 36
-
7/28/2019 Scale Construction
37/111
Exploratory Factor Analysisempirical structure should match the
theoretically expected structure1st case:
ISSAS 2009 - scale construction 37
items oa on one speci ic actor su imensions factors can be correlated)
ISSAS 2009 - scale construction 37
-
7/28/2019 Scale Construction
38/111
(Sub-) Dimensions
ISSAS 2009 - scale construction 38ISSAS 2009 - scale construction 38
-
7/28/2019 Scale Construction
39/111
Exploratory Factor Analysisempirical structure should match the
theoretically expected structure2nd case:
ISSAS 2009 - scale construction 39
Items oa on one common actor an on onespecific factor (facets)
ISSAS 2009 - scale construction 39
-
7/28/2019 Scale Construction
40/111
Facets
ISSAS 2009 - scale construction 40ISSAS 2009 - scale construction 40
-
7/28/2019 Scale Construction
41/111
Confirmatory Factor Analysistheory driven
Model testing
structure is known beforehand
ISSAS 2009 - scale construction 41
test if assumed factor structure explainsobserved variance-covariance matrix
structural equation modeling software
needed (AMOS, Mplus, LISREL)
Bollen (1989)
ISSAS 2009 - scale construction 41
-
7/28/2019 Scale Construction
42/111
simple example
LOVE
affection
love
ISSAS 2009 - scale construction 42ISSAS 2009 - scale construction 42
JOY
security
joy
fortune
happin.
Conent.
C it i f C fi t F t
-
7/28/2019 Scale Construction
43/111
Criteria for Confirmatory Factor
Analysisgoodness of fit:
comparison of the expected and the observedcovariance matrix
?Cov Y Y Cov Y Y =
ISSAS 2009 - scale construction 43
test
RMSEA
comparative fit indices (TLI: Tucker-Lewis Index;
CFI: Comparative Fit Index)
Schermelleh-Engel, Mller, & Moosbrugger (2003)
1 5 1 5 ( , ) ( , )Cov Y Y Cov LOVE JOY =
-
7/28/2019 Scale Construction
44/111
Internal Consistencyestimation:
Cronbachs alpha (should be > .70)
2 2
21
T T
T
k
k
=
ISSAS 2009 - scale construction 44
association of one item with scale
(corrected) item-total correlation (discrimination)
broad constructsmedium rit
narrow construct higher rit
ISSAS 2009 - scale construction 44
-
7/28/2019 Scale Construction
45/111
Reliability of itemsCTT (Classical Test Theory):
iY i = + +
ISSAS 2009 - scale construction 45ISSAS 2009 - scale construction 45
reliability:2
ii
Var( )Rel(Y )
Var( )iY
=
-
7/28/2019 Scale Construction
46/111
Validityitem pool build in such a way that it is content valid
reliability proofedcriterion validity
concurrent: correlation with another measure that shall be
ISSAS 2009 - scale construction 46ISSAS 2009 - scale construction 46
predictedpredictive: prediction (e.g., via regression) of a later event
(test-score)
construct validity
-
7/28/2019 Scale Construction
47/111
Validityconstruct validity
Campbell & Fiske (1959): Multitrait-Multimethod(MTMM) Matrix
conver ent validit
ISSAS 2009 - scale construction 47ISSAS 2009 - scale construction 47
high associations between different measures of the sameconstruct
discriminant validity
trait-method-unit (TMU) more than one method is needed
MTMM analysis
-
7/28/2019 Scale Construction
48/111
MTMM analysis
foundations convergent validity
high associations between different measures of the sameconstruct
discriminant validity
ISSAS 2009 - scale construction 48
Lower associations between measures of differentconstructs
trait-method-unit (TMU)
scores of a measure depend on underlying trait (construct)but also on method
more than one method is needed
disentangle influences due to trait and method
-
7/28/2019 Scale Construction
49/111
Methods?
different raters
self-report, friend, acquaintance teacher, pupils
different tests
ISSAS 2009 - scale construction 49
BDI, Hamilton Scale different kinds of data
cortisol, heart rate, stress rating
productivity rate, ratings of preformance, number ofproduction errors
(measurement occasions)
sub-scales / items
-
7/28/2019 Scale Construction
50/111
Multitrait-Multimethod Matrix
heuristic inspection (Campbell & Fiske, 1959)
ISSAS 2009 - scale construction 50
Monomethod blocks:reliabilities on diagonalHeterotrait-Monomethod triangle
discriminant validity
-
7/28/2019 Scale Construction
51/111
Multitrait-Multimethod Matrix
heuristic inspection (Campbell & Fiske, 1959)
ISSAS 2009 - scale construction 51
heteromethod-blocksconvergent validity coefficients (bold)heterotrait-heteromethod-triangles
-
7/28/2019 Scale Construction
52/111
convergent and discriminant validity
convergent validitycorrelations on validity significantly larger than 0?
discriminant validitycorrelations besides the validity diagonal smaller than on
validity diagonal?
ISSAS 2009 - scale construction 52
monotrait-heteromethod-correlations (MHC) should behigher than than heterotrait correlations of the samevariables
correlations of different traits should be similar acrossmono- and heteromethod blocks (pattern of associations
between traits should be similar across methods)method effects
degree to which monomethod correlations are higher thanheteromethod correlations
-
7/28/2019 Scale Construction
53/111
Critique on C&F approach
correlations depend on reliabilities
What is a high correlations? What are different patterns of correlations
ISSAS 2009 - scale construction 53
w at i erences can e consi ere ig
no statistical model to explain genesis of data
no testable consequences
trait- and method effects are not disentangled
-
7/28/2019 Scale Construction
54/111
Modern MTMM approaches
confirmatory factor analysis (CFA)
CFA-MTMM / SEM-MTMM Models
Separation of true and error components of scores
specification of measurement models
ISSAS 2009 - scale construction 54
a ow or mo e es ng
some allow for a separation of trait- and methodcomponents
Quantification of trait- and method effects possible
factor analysis
-
7/28/2019 Scale Construction
55/111
y
Y1 = 1 + 1 + 1Y
2 = 2 + 2 + 2
Y1 Y2
1 2
ISSAS 2009 - scale construction 55 55
1 2
observed variables (Y1, Y2)
: latent variable (common factor)
1, 2: residualvariablen [measurement error,
uniqueness, unique factors]
loadings i; intercepts
i
factor analysis
-
7/28/2019 Scale Construction
56/111
y
Y1 = 1 + 1 + 1Y
2 = 2 + 2 +
2
Y1 Y2
1 2
ISSAS 2009 - scale construction 56 56
Important indices:
- communality
- reliability
2 ( )( ) ( )
ii
i
VarRel Y Var Y
=
1 2
2 factor model
-
7/28/2019 Scale Construction
57/111
F1:Extraversion
Var(F1) F2:Repair
Var(F2)
Cov(F1, F2)
2 factor model
ISSAS 2009 - scale construction 57 57
E1 E2
1 2
Var(1) Var(2)
R1 R2
3 4
32
Var(3) Var(4)
21 4211
rules of calculations for variances and
-
7/28/2019 Scale Construction
58/111
covariances
E1 = 11F1 + 1 E2 = 21F1 + 2
Var(aX+ bY) = a2 Var(X) + b2 Var(Y) + 2ab Cov(X,Y)
ISSAS 2009 - scale construction 58 58
Cov(a1X1 + b1Y1, a2X2 + b2Y2) =
a1a2 Cov(X1,X2) + a1b2 Cov(X1, Y2) + b1a2 Cov(Y1,X2) + b1b2 Cov(Y1, Y2)
X, Y: variables a, b : constants
2 factor model
-
7/28/2019 Scale Construction
59/111
Var(E1)
Cov(E2, E1) Var(E2)
Cov(R1, E1) Cov(R1, E2) Var(R1)
Cov(R
2,E
1)Cov
(R
2,E
2)Cov
(R
2,R
1) Var(R2)EXTRA1 EXTRA2 REPAIR1 REPAIR2
EXTRA1 0.707
sample-
covariances
ISSAS 2009 - scale construction 59 59
R2
E1
Cov(F1, F2) R2 E2 Cov(F1, F2) R2 R1 Var(F2) R22 Var(F2) + Var(4)
E1
2
Var(F1) + Var(
1)E2
E1
Var(F1) E22 Var(F1) + Var(2)
R1
E1
Cov(F1, F2) R1 E2 Cov(F1, F2) R12 Var(F2) + Var(3)
EXTRA2 0.434 0.584
REPAIR1 0.073 0.073 0.359
REPAIR2 0.113 0.114 0.256 0.320
S
Si l I di t MTMM M d l
-
7/28/2019 Scale Construction
60/111
Single Indicator MTMM Models
Correlated Traits Model
Correlated Traits-Correlated Uniqueness Model Correlated Traits-Uncorrelated Methods Model
ISSAS 2009 - scale construction 60
Correlated Traits Model
-
7/28/2019 Scale Construction
61/111
ISSAS 2009 - scale construction 61
-
7/28/2019 Scale Construction
62/111
ISSAS 2009 - scale construction 62
2
( ) ( ) ( )
( )
jk jk jk
jk jk k jk
jk k jk
Y E
Var Y Var T Var E
Var T E
= +
= +
= +
Implications
-
7/28/2019 Scale Construction
63/111
Implications
systematic variance in observed variables only
caused by traits
correlations between latent variables =discriminantvalidity
ISSAS 2009 - scale construction 63
reliability = consistency (convergent validity) residuals consist of measurement error and method
effects (specific to TMU)
Correlated Traits-Correlated
-
7/28/2019 Scale Construction
64/111
Uniqueness Modell
ISSAS 2009 - scale construction 64
-
7/28/2019 Scale Construction
65/111
ISSAS 2009 - scale construction 652
( ) ( ) ( )
( )
jk jk jk
jk jk k jk
jk k jk
Y E
Var Y Var T Var E
Var T E
= +
= +
= +
Implications
-
7/28/2019 Scale Construction
66/111
Implications
systematic variance in observed variables onlycaused by traits
correlations between latent variables =discriminantvalidity
=
ISSAS 2009 - scale construction 66
residuals consist of measurement error and methodeffects (specific to TMU)
residuals belonging to one method may correlate(method effects generalize)
no separation of effects due to method andmeasurement error
Correlated Traits-Uncorrelated
-
7/28/2019 Scale Construction
67/111
Methods Modell
ISSAS 2009 - scale construction 67
-
7/28/2019 Scale Construction
68/111
ISSAS 2009 - scale construction 68
2 2
( ) ( ) ( ) ( )
( ) ( )
jk jk jk
jk Tjk k Mjk j jk
Tjk k Mjk j jk
Y E
Var Y Var T Var M Var E
Var T Var M E
= +
= + +
= + +
-
7/28/2019 Scale Construction
69/111
consistency :
2
2 2( )( )
( ) ( )
Tjk k
jk
Tjk k Mjk j
Var TCON YVar T Var M
=+
ISSAS 2009 - scale construction 69
Method-specificity:2
2 2
( )( )
( ) ( )
Mjk j
jk
Tjk k Mjk j
Var MMS Y
Var T Var M
=
+
Implications
-
7/28/2019 Scale Construction
70/111
Implications
systematic variance in observed variables caused bytraits and method
each and every method deviates from the trait (nobest method)
ISSAS 2009 - scale construction 70
corre at ons etween atent var a es = scr m nant
validity reliability = consistency (convergent validity) +
method-specificity
residuals consist of measurement error method effects generalize across traits
Correlated Traits-Correlated Methods
-
7/28/2019 Scale Construction
71/111
Minus 1 Modell
ISSAS 2009 - scale construction 71
-
7/28/2019 Scale Construction
72/111
ISSAS 2009 - scale construction 72
2
( ) ( ) ( )
( )
jk jk jk
jk Tjk k jk
Tjk k jk
Y E
Var Y Var T Var E
Var T E
= +
= +
= + 2 2
( ) ( ) ( ) ( )
( ) ( )
jk jk Mjk j jk
jk Tjk k Mjk j jk
Tjk k Mjk j jk
Y M E
Var Y Var T Var M Var E
Var T Var M E
= + +
= + +
= + +
-
7/28/2019 Scale Construction
73/111
consistency :
2
2 2( )( )
( ) ( )
Tjk k
jk
Tjk k Mjk j
Var TCON YVar T Var M
=+
ISSAS 2009 - scale construction 73
Method-specificity:2
2 2
( )( )
( ) ( )
Mjk j
jk
Tjk k Mjk j
Var MMS Y
Var T Var M
=
+
Implications
-
7/28/2019 Scale Construction
74/111
p
one method is gold standard (reference method)
contrast of other methods against reference
trait factor is true-score of reference method (doesnot chan e if methods are added to the model
ISSAS 2009 - scale construction 74
comprises trait and method effects of referencemethod)
method effects are residuals in a latent regression
variance components of trait and method effectsestimable (consistency and method specificity)
Multiple Indicator MTMM-Models
-
7/28/2019 Scale Construction
75/111
p
basis:
TM model choice of model depends on method structure
ISSAS 2009 - scale construction 75
TM-Model
-
7/28/2019 Scale Construction
76/111
Latent variables represent
TMUseparation of systematic and
ISSAS 2009 - scale construction 76
latent variables consist of trait
and method influences
basis for further analysis
Choose among MTMM Models
-
7/28/2019 Scale Construction
77/111
g
method structure
interchangeable raters / methods? students in courses
friends
ISSAS 2009 - scale construction 77
employees (at same level in hierarchy) structurally different raters / methods?
self-, friend, and acquaintances
student and teacher ratings und Lehrerrating group leader and subordinates
Interchangeable Raters
-
7/28/2019 Scale Construction
78/111
g
ANOVA:
Mean is expected value in factor leveldeviations from mean are independent
ISSAS 2009 - scale construction 78
mean
Rating A1
Rating A2
Rating B2
Rating B1
Met. A
Met. B
1
E111
CTUM
-
7/28/2019 Scale Construction
79/111
M12
Y112
M211
1
Y212
T111
1
Y1211
T211
Y111
Y211
M11
M211
M21
11
T211Trait 1
Rater 1
Rater 2
Rater 1
ISSAS 2009 - scale construction 79
M22
T121
Y222
M32Y132
T131
Y232
1Y131
Y231
221
Y122
M231
M221
M31
M231
M221
1
1
Trait 3
Trait 2
Rater 2
Rater 1
Rater 2
Structurally Different Raters
-
7/28/2019 Scale Construction
80/111
rater differ in the quality of their ratings
standard? referenz?
standard is best approximation of true score (Platonic) true-score is trait
ISSAS 2009 - scale construction 80
deviations from this true-score are method-effects
Trait(Trait+Method
of Standard)
Rating A1
Rating A2
Rating B2
Rating B1Met. B
-
7/28/2019 Scale Construction
81/111
TraitRating A1
ISSAS 2009 - scale construction 81
of Standard)Rating A2
Rating B2
Rating B1Met. B
CTC(M-1)Y111
-
7/28/2019 Scale Construction
82/111
M112
T111
Y112
Y211
Y212
Y113
Y221
Y213
Y121
T121
M113
T111
ISSAS 2009 - scale construction 82
Y222
Y122
Y223
Y123
Y232
Y132
Y231
Y131
Y233
Y133
T131
M122
M123
M132
M133
T121
T131
Multiple Indicator MTMM Models
-
7/28/2019 Scale Construction
83/111
separation of measurement error fromsystematic influences
model choice based on method structure
ISSAS 2009 - scale construction 83
determined method factors are trait specific (one method
may overestimate DIF but underestimate DDF)
method factors can be related to externalvariables
CTUM and CTC(M-1) model
-
7/28/2019 Scale Construction
84/111
ISSAS 2009 - scale construction 84
CTUM model
-
7/28/2019 Scale Construction
85/111
ISSAS 2009 - scale construction 85
CTC(M-1) model
-
7/28/2019 Scale Construction
86/111
ISSAS 2009 - scale construction 86
Research (common) practice
-
7/28/2019 Scale Construction
87/111
content validity is generally assured
many authors report Cronbachs , (corrected)item-total correlations
ISSAS 2009 - scale construction 87
exp oratory actor ana ysis are requent y use
confirmatory factor analysis is less frequentlyused
MTMM models are found in larger researchprograms
87ISSAS 2009 - scale construction
Norms
-
7/28/2019 Scale Construction
88/111
Scales of measurement for most constructs isarbitrary
Meaning of scores can only be determined in relationto some frame of reference
ISSAS 2009 - scale construction 88
(across and within individuals) as frame large sampleRepresentative sample (multiple samples)
descriptive statistics of scale scores
Norms for subpopulations (male and female; students andnon-students samples; ethnical backgrounds; differentcultures sharing the same language)
Outlook
-
7/28/2019 Scale Construction
89/111
Computer programs
Dichotomous / ordinal data Cross-cultural research
ISSAS 2009 - scale construction 89
Longitudinal data
Multilevel MTMM models
89ISSAS 2009 - scale construction
Computer Programs
-
7/28/2019 Scale Construction
90/111
Simple statistics, Cronbachs , item total
correlations, exploratory factor analysis:
SPSS
R
ISSAS 2009 - scale construction 90
S-Plus
Confirmatory Factor Analysis (SEM)
Mplus
LISREL
AMOS
EQS
Dichotomous / Ordinal Data
-
7/28/2019 Scale Construction
91/111
Graded Response Model (Samejima, 1969) The observed variable Yijk is linked to the item-specific probit
variable by the following probability function:
( ) 2
21
|
ijk isjk x
P Y s e dx
= =
ISSAS 2009 - scale construction 91
The parameter isjk is a difficulty parameter for each responsecategory bound s (s > 0) for item imeasuring traitjwithmethod k, and represents the probability distribution of the
standard normal distribution.
2s
0.8
1
greatProbabilityto chooseat least category s
( )|ijk ijk P Y s
( )1|ijk ijk P Y
-
7/28/2019 Scale Construction
92/111
( ) ( )( ) 2
21
|2
ijk isjk x
ijk ijk ijk isjk P Y s e dx
= =
0
0.2
0.4
0.6
-6 -5 -4 -3 -2 -1 0 1 2 3
unpleasant pleasant
ijk
( )
( )4 |ijk ijk P Y
( )3 |ijk ijk P Y ( )2 |
ijk ijk P Y
ISSAS 2009 - scale construction 92
0
0.2
0.4
0.6
0.8
1
-5 -4 -3 -2 -1 0 1 2 3
0 -not at all
1 2 3
very much so -4
response probability
pleasantunpleasant
good( )|ijk ijk P Y s =
ijk
( ) ( ) ( )| | 1|ijk ijk ijk ijk ijk ijk P Y s P Y s P Y s = = +
( ) ( ) ( )3 | 3 | 4 |ijk ijk ijk ijk ijk ijk P Y P Y P Y = =
Factor Analysis for Ordered Categorical Data
-
7/28/2019 Scale Construction
93/111
latent continuous variable, causes observed responses
*
ijkY
Assumption:
ISSAS 2009 - scale construction 93
( )*
1
*
( 1)
*
1
1, for
, for
0, for
ijkijk ijk c ijk
ijk sijk ijk s ijk
ijk ijk
c Y
Y s Y
Y
+