![Page 1: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/1.jpg)
Experimental, Quasi-experimental, and Single Subject Research
774/801 Sept 1, 2004
John Hattie & Tony Hunt
![Page 2: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/2.jpg)
It is simple: There is no perfect experiment
in education
There is nearly always a trade off between the
Power to generalise - from sample to population- from items to the behaviour domain- from conditions in the study to all intended conditions
and the
Power to convince- there are many audiences
PG PC
![Page 3: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/3.jpg)
Power to Generalise
How confident can be generalise from the study to all “similar” situations
Is the design replicable/reproducible/exchangeable?
Is the evidence/conclusions unique to this study?
Have the generalisations taken into account all possible competing views – plausible alternative rival explanations (PARE)
PG
PC
![Page 4: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/4.jpg)
Power to convince
Who are we trying to convince If it is a colleague(s) then more situation
specificity may be convincing (kids/classrooms/schools like mine)
If it is the educational community, then situation needs to be less critical
PC
PG
![Page 5: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/5.jpg)
Resolution: Linking Power
Experimental design consists of a series of links:– It is as strong as the weakest link– Each link influences the next link– Desirable to have equal strength– Does each link have explanatory power– Are conclusions credible to the intended audience
![Page 6: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/6.jpg)
History
Stanley and Campbell (1963) Cook and Campbell (1979) Shadish, Cook & Campbell (2002)
Evidence based
All based on designing studies that can lead to explanation and claims of causality
![Page 7: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/7.jpg)
Explanation and Cause
1. Cause and effect must be related
(e.g., self-concept & achievement)
1. There needs to be temporal order (cause before effect)
2. Need to rule out other explanations/ other Plausible Alternative Rival Explanations (PARE)
![Page 8: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/8.jpg)
Campbell & Stanley (1963)
Pretest-Posttest Control Group Design
Pre Treatment Post
R O X O
R O O
Randomisation – aiming for representativeness
![Page 9: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/9.jpg)
But can we randomise
No Child Left Behind Tennessee Class Size Study
![Page 10: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/10.jpg)
Quasi-experimentation:
When you do not have so much control over allocation of treatment, conditions, sample
When you have non-equivalent groups
In quasi-experimentation, the researcher has to enumerate alternative explanations one by one, decide which are plausible, and then use logic, design, and measurement to assess whether each one is operating in a way that might explain any observed effect (Shadish, Cook & Campbell, 2002, p. 14)
Relates to the Popper notion of falsification: What evidence would you accept that you are wrong?
![Page 11: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/11.jpg)
Examples of Quasi-experimental designs
Divorce LawsOzdowski, S.A. & Hattie, J.A. (1981).
The impact of divorce laws on divorce rate in Australia: A time series analysis. Australian Journal of Social Issues, 16, 3-17.
0
2
4
6
8
10
12
14
16
18
20
1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981
a. Time SeriesO1 O2 O3 O4 O5 X O6 O7 O8
O9
![Page 12: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/12.jpg)
ABA designA B A
O1 O2 O3 X4 X5 X6 O7 O8 O9 Le Fevre, et.al. (2002). Adequate Decoders
-10
0
10
20
30
40
50
60
70
80
90
100
0 10 20 30 40
Sessions
School 3
-10
0
10
20
30
40
50
60
70
80
90
100
0 10 20 30 40
Generalization
BLine 1 BLine 2 Intervention FollowUp
Manitenance
Daily Mean
School 1
-10
0
10
20
30
40
50
60
70
80
90
100
0 40
School 2
![Page 13: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/13.jpg)
Multilevel Design: Hierarchical Linear Modelling
Students within classes within schools
E.g., Tracking/Streaming
School 1 School 2…
Teacher 1 Teacher 2
Class 1 Class 2 Class 1 Class 2
![Page 14: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/14.jpg)
Structural Equation Modelling
Nutrition
nutr138e5
.74nutr123e4
.78nutr120e3.84
Exercise
exer53
e6
.62exer54e7
.54exer55e8
.85
Self Care
selfc59
e14
.57
selfc58
e13
.79
selfc57
e12
.66
selfc56
e11
.68
exer131e9 .88
exer139e10
.76
nutr51e2 .68
nutr50e1
.84
.79
.69
.80
Structural Model for Physical Self
Social
jfriend e15.97
jlove e16
.92
Interactive
jintell e17
.93
jwork e18.89
jemot e19.92
jcontr e20
.94
jhumor e21.86
-.23
.54
.60
e22
e23
-.14.55
.67
![Page 15: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/15.jpg)
Minimal requirements for Studies
Sampling– Items to behaviour domains– People to all possible people– Conditions to all possible conditions
Representative sampling via– Random sampling– Stratified random sampling
![Page 16: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/16.jpg)
Variables
At the end of your study, can I say “Aha, so that is what you mean, now I am clear”
Open constructs NOT Definitions No such thing as immaculate perception Dependent - Manipulable Independent - Nonmanipulable
![Page 17: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/17.jpg)
Dependability
How reliable/consistent/replicable are your measures/ observations
![Page 18: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/18.jpg)
Validity = Interpretations
Validity - "an integrated evaluative judgement of the degree to which empirical evidence and theoretical rationales support the adequacy and appropriateness of inferences and actions based on test scores or other modes of assessment".
Not validity of a test, but validity of interpretations
![Page 19: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/19.jpg)
Validity of your study …
Is related to having ruled out
Plausible Alternative Rival Explanations (PARE)
CONTROL CONTROL CONTROL
… some examples
![Page 20: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/20.jpg)
1. PARE: Power
Is your study POWERFUL enough to detect the effect you are investigating
![Page 21: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/21.jpg)
1. PARE: Power
Is your study POWERFUL enough to detect the effect you are investigating
Do chickens have lips?
![Page 22: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/22.jpg)
![Page 23: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/23.jpg)
2. PARE: Chance
Did the effect/conclusion occur by chance
E.g., That two means are the same – the
hypothesis of no difference
Setting a rejection level, say =.05
![Page 24: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/24.jpg)
3. PARE: Type II errors
Type I errors – Rejecting a claim when it is true (=.05)
Type II errors – Accepting a claim when it is false (e.g., chickens do not have lips, if it is indeed true)
![Page 25: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/25.jpg)
4. PARE Reliability of your measures
If the reliability is low, then the scores “wobble” and no guarantee you will get same results using these instruments (tests, observations, interviews, etc.)
Was the treatment “consistent” in the various classes/implementations?
![Page 26: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/26.jpg)
5. PARE: Was the treatment implemented?
Degree of implementation The Hong Kong Practical Science Study
(Cheung, Hattie, & Bucat, 1997)
![Page 27: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/27.jpg)
6. PARE: Maturation
Showing change may not be enough as kids improve anyway (e.g., by maturation)
Method to measure change = Effect-sizesPost-Pre/spread = Effect-size
X2 – X1 sddiff
e.g., Before = 12, After = 15, spread = 6 15-12 = .5
6
![Page 28: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/28.jpg)
0
5000
10000
15000
20000
25000
< -1
.01 -.98-.9
3-.8
8-.8
3-.7
8-.7
3-.6
8-.6
3-.5
8-.5
3-.4
8-.4
3-.3
8-.3
3-.2
8-.2
3-.1
8-.1
3-.0
8-.0
3 0.0
7.1
2.1
7.2
2.2
7.3
2.3
7.4
2.4
7.5
2.5
7.6
2.6
7.7
2.7
7.8
2.8
7.9
2.9
71.
021.
071.
121.
171.
221.
271.
321.
371.
421.
471.
521.
571.
621.
671.
721.
771.
821.
871.
921.
97>2
.0
Distribution of effects
Zero achievement
Average effect
![Page 29: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/29.jpg)
0
5000
10000
15000
20000
25000
< -1
.01 -.98-.9
3-.8
8-.8
3-.7
8-.7
3-.6
8-.6
3-.5
8-.5
3-.4
8-.4
3-.3
8-.3
3-.2
8-.2
3-.1
8-.1
3-.0
8-.0
3 0.0
7.1
2.1
7.2
2.2
7.3
2.3
7.4
2.4
7.5
2.5
7.6
2.6
7.7
2.7
7.8
2.8
7.9
2.9
71.
021.
071.
121.
171.
221.
271.
321.
371.
421.
471.
521.
571.
621.
671.
721.
771.
821.
871.
921.
97>2
.0
Distribution of effects
Maturation
![Page 30: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/30.jpg)
The disasters …
71 programmed instruction 801 .14
72 finances 1634 .14
73 problem based learning 41 .12
74 diet 255 .12
75 gender (female-male) 9020 .09
76 inductive teaching 570 .06
77 team teaching 41 .06
78 ability grouping 3355 .05
79 class size 2559 .05
80 open vs. traditional 3426 -.01
81 summer vacation 269 -.06
82 retention 3626 -.17
83 transfer of school 354 -.26
84 disruptive students 1511 -.78
![Page 31: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/31.jpg)
The also rans …56 metacognitive intervention 921 .29
57 math programs 3326 .27
58 audio-visual 2699 .26
59 gifted programs 47 .25
60 coaching 1076 .24
61 behavior objectives 157 .24
62 calculators 238 .24
63 mainstreaming 1641 .21
64 questioning 493 .20
65 learning hierarchies 168 .19
66 attitude to math 1122 .19
67 desegregation 1590 .18
68 play 129 .16
69 television 4337 .15
![Page 32: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/32.jpg)
Almost there …
42 tutoring 136 .35
43 activity-based programs 674 .35
44 remedial programs 1438 .35
45 classroom climate 2726 .35
46 social skills training 5472 .35
47 time 1680 .34
48 CAI 18231 .32
49 inquiry based teaching 2740 .32
50 preschool 242 .32
51 whole language 198 .31
52 within class grouping 2359 .31
53 testing 1463 .31
54 problem solving 1141 .30
55 background 692 .30
![Page 33: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/33.jpg)
In the middle …
29 parent involvement 2597 .46
30 bilingual programs 1501 .46
31 adjunct aids 659 .45
32 concept mapping 18 .45
33 advance organizers 2106 .44
34 hypermedia instruction 317 .44
35 socio economic status 1657 .44
36 perceptual-motor skills 7592 .42
37 individualised instruction 5948 .42
38 homework 568 .41
39 competitive learning 144 .41
40 simulations 972 .37
41 expectations 912 .36
![Page 34: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/34.jpg)
Worth having …
14 self-assessment 152 .54
15 mastery learning 1933 .53
16 creativity programs 2340 .52
17 interactive video 1152 .52
18 psycho-linguistics 4404 .51
19 goals 959 .51
20 peer influence 366 .50
21 early intervention 30971 .49
22 outdoor education 294 .49
23 science 4124 .49
24 inservice ed 18644 .48
25 acceleration 371 .47
26 motivation 2196 .47
![Page 35: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/35.jpg)
The MAJOR Influences …
Influence # effects Mean
1 Direct instruction 1925 .93
2 Reciprocal teaching 52 .86
3 Feedback 13209 .81
4 Cognitive strategy training 7649 .80
5 Classroom behaviour 361 .71
6 Prior achievement 2094 .71
7 Phonological awareness 2630 .70
8 Home encouragement 25706 .69
9 Piagetian programs 786 .63
10 Cooperative learning 1153 .59
11 Reading programs 14945 .58
12 Quality of teaching 808 .55
13 Study skills 3224 .54
![Page 36: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/36.jpg)
Identifying that what matters
Percentage of Achievement Variance
Students
Teachers
Home
PeersSchools Principal
![Page 37: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/37.jpg)
7. PARE: Testing
People become test wise and/or may respond different when under test conditions
White space and testing in asTTle Testwiseness
![Page 38: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/38.jpg)
Test of Objective EvidenceEach of the questions in the following set has a logical or
"best" answer from its corresponding multiple choice answer set. Please record your eight answers.
1. The purpose of the cluss in 2 Trassig is true when furmpaling is to removeA clump trasses the vonA cluss-prags B the viskal flans, if the viskal is B tremails donwil or zortilC cloughs C the belgo frulsD pluomots D dissels lisk easily
3 The sigia frequently overfesks the 4. The fribbled breg will minter best trelsum because with an
A all sigias are mellious A derstB all sigias are always votial B morstC the trelsum is usually tarious C sortarD no trelsa are feskable D ignu
![Page 39: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/39.jpg)
Test of Objective Evidence, Part II
5 The reasons for tristal doss are 6 Which of the following is/are always present when trossels
are being gruven?
A the sabs foped and the doths tinzed A rint and vostB the dredges roted with the crets B vostC few rakobs were accepted in sluth C shum and vostD most of the polats were thonced D vost and plone
7 The mintering function of the ignu is most 8effectively carried out in connection with
A a razma toi AB the groshing stantol BC the fribbled breg CD a frailly sush D
![Page 40: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/40.jpg)
8. PARE Statistical Regression
When taking extreme groups the means tend to move to the middle.
Why do the tallest fathers have shorter sons, and the shortest fathers have taller sons?
![Page 41: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/41.jpg)
…. Regression to the Mean
Special Education (e.g., Sesame Street) Effective schools Gifted education
![Page 42: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/42.jpg)
9. PARE Response rates
The returns of questionnaires/tests/interviews
should be high
What is typical?
![Page 43: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/43.jpg)
Meta-analyses of Response Rates
Typical return is 50%
Three major factors:
1. Salience (77% vs 42%)
2. Number of follow ups (halve each time)
3. Lack of clutter/ orderliness
Not length (ave 7 pages, 72 questions), colour,
![Page 44: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/44.jpg)
10. Change scores
The difference between post-pre scores
Problems
1 Unreliable
2 Are you measuring same thing both times
3 Regression to the mean
![Page 45: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/45.jpg)
11 PARE: Experimenter effects
Hawthorne effect: Because we know we are in an experiment this alters our responses
Hans the Horse Pygmalion in the classroom Christine Rubie’s thesis Stanley Milgrim’s experiment
![Page 46: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/46.jpg)
12. PARE: Restriction of range
When you choose/focus on a narrow range of abilities (etc.) this can be misleading
Picture …
![Page 47: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/47.jpg)
0
2
4
6
8
10
12
14
16
18
20
0 5 10 15 20 25
![Page 48: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/48.jpg)
0
2
4
6
8
10
12
14
16
18
20
0 5 10 15 20 25
![Page 49: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/49.jpg)
13. PARE: Specification of target and accessible sample/population
Most experiments are highly local but have general aspirations
Often, there are two groups you are generalising to: e.g., all secondary students in NZ, and to all secondary students you have access -- from which to sample
![Page 50: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/50.jpg)
![Page 51: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/51.jpg)
14: Interactions
The model of individual differences indicates that we should modify our teaching methods to allow for individual differences in the class
0
5
10
15
20
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Competitive Learning
Cooperative Learning
Girls Boys
![Page 52: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/52.jpg)
The art of research design is to devise experiments to identify the explanation and cause of effects – by
Maximising the chance that the conclusions are defensible and
Minimising the PAREs
Such that you have
Power to Generalise and
Power to Convince
![Page 53: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/53.jpg)
Unobtrusive measures
Which painting do most people watch? Friendship in cities Racism in suburbs/cities
![Page 54: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/54.jpg)
Statistical Methods to assist …
Correlation Analysis of variance (anova) Cross-tabulation
![Page 55: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/55.jpg)
![Page 56: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/56.jpg)
![Page 57: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/57.jpg)
Comparing means: Magnitude and Chance
Magnitude Effect-sizes Chance Analysis of Variance
![Page 58: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/58.jpg)
Well-being
What are the differences in levels of WELL-BEING among males and females, and between Australia and New Zealand
Country * GENDER Cross tabulationCount GENDER
MALE FEMALE Total
Country New Zealand 516 644 1160Australia 421 694 1115
Total 937 1338 2275
![Page 59: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/59.jpg)
Australia Mn sd Effect-size
Male 45.7 10.6
Female 46.2 10.6 0.04
Total 46.0 10.6
New Zealand
Male 53.6 7.5
Female 54.3 7.4 0.08
54.0 7.5
NZ - Australia 0.89
![Page 60: Experimental, Quasi- experimental, and Single Subject Research 774/801 Sept 1, 2004 John Hattie & Tony Hunt](https://reader035.vdocument.in/reader035/viewer/2022062219/551bea60550346b4588b62b3/html5/thumbnails/60.jpg)
anova
Source df MS F p
Country 1 35211.9 416.71 <.001
Gender 1 151.8 1.80 .180
Country * Gender 1 6.1 0.07 .787
Error 2271 84.5