lecture 8 – internal validity threats continued
DESCRIPTION
Lecture 8 – Internal Validity Threats continued. EPSY 640 Texas A&M University. TESTING. PRETESTING OR REPEATED TESTING proactive or retroactive interference memory change in cognitive schema or structure. TESTING. SOLOMON 4 GROUP DESIGN:. 1 R O X O 2 R O O 3 R X O - PowerPoint PPT PresentationTRANSCRIPT
Lecture 8 – Internal Validity Threats continued
EPSY 640
Texas A&M University
TESTING
• PRETESTING OR REPEATED TESTING– proactive or retroactive interference– memory– change in cognitive schema or structure
TESTING
SOLOMON 4 GROUP DESIGN:
1 R O X O2 R O O3 R X O4 R O
TESTING
• Willson and Putnam (1982) conducted a meta-analysis of studies using pretests and concluded that pretesting can effect both cognitive and psychological testing. The largest effects can be expected to occur within about two weeks of pretesting, and effects on achievement and intelligence tests is expected to be greater than effects for attitude or psychological tests.
MORTALITY
• LOSS OF CASES FROM GROUPS NONRANDOMLY
• POTENTIAL FOR PARTICULAR CONDITION TO CAUSE DROPOUTS
• NEED FOR INITIAL INFORMATION ON ALL PARTICIPANTS TO EXAMINE CHARACTERISTICS OF DROPOUTS
REGRESSION TOWARD MEAN
• EQUIVALENT TO IMPERFECT CORRELATION BETWEEN VARIABLES
• EXISTS WHENEVER A SAMPLE IS SELECTED AND ITS MEAN IS NOT EQUAL TO THE POPULATION MEAN, THEN MEASURED ON ANOTHER VARIABLE
REGRESSION TOWARD MEAN
• REGRESSION OCCURS WITH:
• GIFTED AND TALENTED RESEARCH
• RESEARCH IN SPECIAL EDUCATION
• RESEARCH ON ANY NONREPRESENTATIVE GROUPS
REGRESSION TOWARD MEAN AND MATCHING
• MATCHING ALWAYS PRODUCES REGRESSION EFFECTS UNLESS THE TWO SAMPLES MATCHED HAVE THE SAME POPULATION MEANS
REGRESSION TOWARD MEAN AND MATCHING
• In a study of a new intelligence test researchers gave the test to students who had been previously evaluated with another intelligence test and found to have scored over 120, qualifying them for gifted and talented status. The researchers found that the students averaged below 120 on the new test and concluded that the new test did not adequately assess gifted and talented status. What they actually found was the regression effect due to the lack of perfect correlation between the two intelligence tests. A simple calculation from their reported results conformed quite well to the expected lowering of score from one assessment to the other.
Statistical regression due to matching
matching mean
for
selecting
variable
group 1 group 2
+1SD
+ ½ SD
Outcome correlates .7 with selecting variable:
Group 1 predicted score on outcome = +.7SD
Group 2 predicted score on outcome = +.35 SD
Thus, .35SD difference has been created with no treatment
HISTORY
• CHANGES AND EVENTS OUTSIDE TREATMENT CONDITIONS THAT AFFECT OUTCOMES– Local or national events of stress– Weather– Political change
HISTORY• Campbell and Ross (1968) and Glass (1968) evaluated the effect of a speed
limit law in Connecticut enacted to begin in January, 1955. Politicians claimed within several months that the law had reduced significantly auto deaths, using data from the previous several years to support their claim. Indeed, statistical analysis supported a significant drop in deaths in the months following the implementation of the law, even accounting for typical drops occurring after the Christmas season. Glass (1968) noted that an analysis of the surrounding states showed a similar drop even though no laws had been enacted in them. An investigation of weather conditions at the time showed that a series of blizzards at the time severely restricted driving, particularly high speed driving that would have contributed otherwise to auto deaths. He and Campbell and Ross (1968) discounted the legal experimental account in favor of the historical explanation. Willson (1973), reported in Glass, Willson, and Gottman (1975) demonstrated that the legal effect was not significant when adjusted for the comparable effects in other adjacent states
MONTH
103.00
97.00
91.00
85.00
79.00
73.00
67.00
61.00
55.00
49.00
43.00
37.00
31.00
25.00
19.00
13.00
7.00
1.00
Co
nn
ect
icu
lt F
ata
litie
s p
er
10
0,0
00
Dri
ver
Mile
s, 1
95
1-6
0
700
600
500
400
300
200
100
MONTH
103.00
97.00
91.00
85.00
79.00
73.00
67.00
61.00
55.00
49.00
43.00
37.00
31.00
25.00
19.00
13.00
7.00
1.00
Ma
ssa
chu
sett
s F
ata
litie
s p
er
10
0,0
00
Dri
ver
Mile
s, 1
95
1-6
0
700
600
500
400
300
200
100
MONTH
103.00
97.00
91.00
85.00
79.00
73.00
67.00
61.00
55.00
49.00
43.00
37.00
31.00
25.00
19.00
13.00
7.00
1.00
Ne
w J
ers
ey
Fa
talit
ies
pe
r 1
00
,00
0 D
rive
r M
iles,
19
51
-60
700
600
500
400
300
200
100