Types of Validity

Content Validity

Criterion Validity

Construct Validity

Predictive Validity

Concurrent Validity

Convergent Validity

Discriminant Validity

Adapted from Sekaran, 2004

• Construct Validity• Extent to which hypotheses about construct are

supported by data1. Define construct, generate hypotheses about

construct’s relation to other constructs

2. Develop comprehensive measure of construct & assess its reliability

3. Examine relationship of measure of construct to other, similar and dissimilar constructs

• E.g. Height & Weight; Networking & career outcomes study; Learning Style Orientation measure

Types of Validity

• Multi-trait multi-method matrix• Convergent validity coefficient

• Absolute size of correlation between different measures of the same construct should be large & significantly different from zero

• Discriminant validity coefficient• Correlations between the same construct measured by

different methods should be significantly different from correlations of

• Different constructs measured by different methods (across methods & across constructs)

• Different constructs measured by same method (method bias)

Establishing Construct Validity

 

O-H SR-H O-W SR-W

O-H 1.00

SR-H .98 1.00

O-W .55 .56 1.00

SR-W .68 .69 .92 1.00

Convergent & Discriminant ValidityCorr b/w Objective (O) & Self-Reports (SR) of Height &

Weight

• Convergent validity (across methods)

• Objective and subjective measures of height are correlated .98

• Objective and subjective measures of weight are correlated .92

Convergent & Discriminant Validityof Objective & Self-Reports of Height & Weight

• Discriminant validity (across constructs)• Objective measures of height & weight are corr .55• Subjective measures of height & weight are corr.69

• STRONG CASE: Are the correlations b/w the same construct measured by different methods significantly different from corr b/w different constructs measured by same methods

• i.e., Are .92 & .98 significantly different from .55 & .69?

• Convert rs to z scores and compare

Discriminant Validityof Objective & Self-Reports of Height & Weight

• Discriminant validity (across constructs)• Objective height & subjective weight are corr .68• Subjective height & objective weight are corr .56

• WEAK CASE: Are the correlations b/w the same construct measured by different methods significantly different from corr b/w different constructs measured by different methods

• i.e., Are .92 & .98 significantly different from .56 & .68

Discriminant Validityof Objective & Self-Reports of Height & Weight

• For convergent validity, different measures of the same construct should be highly correlated

• Note: In networking study, diff measures= diff subscales

– Corr b/w diff measures of career success (promotion, salary, perceived career success) range from .20 to .36 (moderate support)• .20 & .36 should not be sig diff from each other

– Corr b/w diff measures of networking (increasing internal visibility, socializing etc.) range from .03 to .43 (weak support)• .03 & .43 should not be sig diff from each other

Establishing Construct Validity in Networking Study

• For discriminant validity, different measures of the same construct should be more highly correlated than different measures of different constructs– Correlations b/w career success & networking (.01

to .35) should be sig different from• Corr b/w diff measures of career success (.20 to .36)

• Corr b/w diff measures of networking (.03 to .43)

Establishing Construct Validity in Networking Study

• Developed items by generating critical incidents (Study 1)– N=67– Yes/no responses to statements– Recall of learning events

• Two types of learning: theoretical, practical (2)

• Two types of outcomes=success, failure (2)

• 2 x 2 events per participant

• 112 items constructed in total

Learning Style Study

• Study 1 Part 2:

• Created 112 items from critical incidents– Administered to 154 participants

– 5-point likert scale (agree/disagree)• Extracted 5 factor solution w/factor analyses

• 54 items loaded highly on the 5 factors

• Content validity: item sorting by 8 grad students

– Also administered personality scale

Learning Style Study

• Item Development Study (study 1) • Convergent Validity

• High reliabilities of subscales of Learning Style (.81-.91)

• Corr b/w different measures (subscales) of learning style =.01 to.32 but 1 only corr is significant

– Weak support for convergent validity of new learning style measure

• Discriminant validity• Corr b/w different measures of different constructs

(Learning Style & personality) .42 to .01 should be lower than and sig diff from corr b/w different measures of same construct (subscales of learning style) .01 to .32

Learning Style Study

• Validation Study (study 2)– N=350 -193– New learning style, Personality, old Learning

style, preferences for instructional & assessment methods

– Construct validity• Confirmatory factor analysis confirms 5

dimensions• Reliability of new learning style subscales=.74

to .87 compared to…– Reliability of old learning style subscales=.83 to .86– Reliability of personality subscales=.86 to .95

Learning Style Orientation Measure

• Validation Study (study 2)– Convergent validity

• Corr b/w similar measures of key construct =Corr b/w diff subscales of new learning style 01 to .23 should be comparable to…

– Corr b/w similar measures of other constructs in the study

• Diff subscales of old learning style .23 to .40

• Diff subscales of personality .01 to .27

Learning Style Orientation Measure

• Validation Study (study 2)– Discriminant validity

• Corr b/w measures of similar constructs= Corr b/w new learning style subscales & old learning style = .01 to .31

• Corr b/w measures of different constructs – Corr b/w new learning style & personality subscales is

.01 to .55

– Corr b/w old learning style & personality subscales= .02 to .38

Learning Style Orientation Measure

• Validation Study– Incremental validity (aka construct validity)

• Additional variance explained by old vs. new learning style measures in preferences for assessment & instruction

Learning Style Orientation Measure

DV LSOM (new)

LSI (old)

Subjective assessment .15 .01

Interactional instruction .21 .04

Informational instruction .06 .00

Types of Validity

Content Validity

Criterion Validity

Construct Validity

Predictive Validity

Concurrent Validity

Convergent Validity

Discriminant Validity

Adapted from Sekaran, 2004