avoiding self-delusion: new technologies and expectation effects · 2005. 7. 27. · 1. the placebo...
TRANSCRIPT
![Page 1: Avoiding Self-Delusion: New Technologies and Expectation Effects · 2005. 7. 27. · 1. The Placebo and Halo Effects • Placebo Effect: a technology has an effect, because the person](https://reader036.vdocument.in/reader036/viewer/2022081623/613efa14c500cf75ab363ab7/html5/thumbnails/1.jpg)
Avoiding Self-Delusion: New Technologies and
Expectation EffectsRuben R. Puentedura, Ph.D.
![Page 2: Avoiding Self-Delusion: New Technologies and Expectation Effects · 2005. 7. 27. · 1. The Placebo and Halo Effects • Placebo Effect: a technology has an effect, because the person](https://reader036.vdocument.in/reader036/viewer/2022081623/613efa14c500cf75ab363ab7/html5/thumbnails/2.jpg)
Four Types of Expectation Effects
![Page 3: Avoiding Self-Delusion: New Technologies and Expectation Effects · 2005. 7. 27. · 1. The Placebo and Halo Effects • Placebo Effect: a technology has an effect, because the person](https://reader036.vdocument.in/reader036/viewer/2022081623/613efa14c500cf75ab363ab7/html5/thumbnails/3.jpg)
1. The Placebo and Halo Effects
• Placebo Effect: a technology has an effect, because the person interacting with that technology believes it will.
• Halo Effect: one aspect of a technology colors how other aspects of that technology are perceived, and consequently its effect.
![Page 4: Avoiding Self-Delusion: New Technologies and Expectation Effects · 2005. 7. 27. · 1. The Placebo and Halo Effects • Placebo Effect: a technology has an effect, because the person](https://reader036.vdocument.in/reader036/viewer/2022081623/613efa14c500cf75ab363ab7/html5/thumbnails/4.jpg)
2. The Hawthorne Effect
• The fact that someone is exposed to a technological shift by itself can affect how they perform, regardless of the specifics of that technological shift.
![Page 5: Avoiding Self-Delusion: New Technologies and Expectation Effects · 2005. 7. 27. · 1. The Placebo and Halo Effects • Placebo Effect: a technology has an effect, because the person](https://reader036.vdocument.in/reader036/viewer/2022081623/613efa14c500cf75ab363ab7/html5/thumbnails/5.jpg)
3. The John Henry Effect
• A group that is not exposed to a technological shift, and knows that another one is, may view itself as in competition with the latter group and change its performance accordingly.
![Page 6: Avoiding Self-Delusion: New Technologies and Expectation Effects · 2005. 7. 27. · 1. The Placebo and Halo Effects • Placebo Effect: a technology has an effect, because the person](https://reader036.vdocument.in/reader036/viewer/2022081623/613efa14c500cf75ab363ab7/html5/thumbnails/6.jpg)
4. The Pygmalion Effect
• Teachers' expectations of the performance of their students will tend to determine their actual performance.
![Page 7: Avoiding Self-Delusion: New Technologies and Expectation Effects · 2005. 7. 27. · 1. The Placebo and Halo Effects • Placebo Effect: a technology has an effect, because the person](https://reader036.vdocument.in/reader036/viewer/2022081623/613efa14c500cf75ab363ab7/html5/thumbnails/7.jpg)
Controlling for Expectation Effects
![Page 8: Avoiding Self-Delusion: New Technologies and Expectation Effects · 2005. 7. 27. · 1. The Placebo and Halo Effects • Placebo Effect: a technology has an effect, because the person](https://reader036.vdocument.in/reader036/viewer/2022081623/613efa14c500cf75ab363ab7/html5/thumbnails/8.jpg)
Tools for Research
• Try to have multiple instructors use the technology
• Try to have multiple classes use the technology
• Have a third party observe instructors and classes
• Look for trends in class performance throughout the term
• Use end-of-term attitudinal surveys
![Page 9: Avoiding Self-Delusion: New Technologies and Expectation Effects · 2005. 7. 27. · 1. The Placebo and Halo Effects • Placebo Effect: a technology has an effect, because the person](https://reader036.vdocument.in/reader036/viewer/2022081623/613efa14c500cf75ab363ab7/html5/thumbnails/9.jpg)
Experimental Design
GroupsTeacher Expectation
For Benefit For No Benefit
1. No-treatment control
A B
2. Hawthorne control
C D
3. Experimental procedure
E F
Essential
Important
Desirable
![Page 10: Avoiding Self-Delusion: New Technologies and Expectation Effects · 2005. 7. 27. · 1. The Placebo and Halo Effects • Placebo Effect: a technology has an effect, because the person](https://reader036.vdocument.in/reader036/viewer/2022081623/613efa14c500cf75ab363ab7/html5/thumbnails/10.jpg)
Cohen’s Effect Size Index d
mA, mB: mean scores for the two groups being compared
σA, σB: standard deviation of the scores for the two groups being compared
d =|mA −mB |√
σ2A+σ2
B2
![Page 11: Avoiding Self-Delusion: New Technologies and Expectation Effects · 2005. 7. 27. · 1. The Placebo and Halo Effects • Placebo Effect: a technology has an effect, because the person](https://reader036.vdocument.in/reader036/viewer/2022081623/613efa14c500cf75ab363ab7/html5/thumbnails/11.jpg)
Small Effect Size
Scores
Frequency
−4 −2 0 2 4
05
1015
20
d=0.2
![Page 12: Avoiding Self-Delusion: New Technologies and Expectation Effects · 2005. 7. 27. · 1. The Placebo and Halo Effects • Placebo Effect: a technology has an effect, because the person](https://reader036.vdocument.in/reader036/viewer/2022081623/613efa14c500cf75ab363ab7/html5/thumbnails/12.jpg)
Medium Effect Size
Scores
Frequency
−4 −2 0 2 4
05
1015
20
d=0.5
![Page 13: Avoiding Self-Delusion: New Technologies and Expectation Effects · 2005. 7. 27. · 1. The Placebo and Halo Effects • Placebo Effect: a technology has an effect, because the person](https://reader036.vdocument.in/reader036/viewer/2022081623/613efa14c500cf75ab363ab7/html5/thumbnails/13.jpg)
Large Effect Size
Scores
Frequency
−4 −2 0 2 4
05
1015
20
d=0.8
![Page 14: Avoiding Self-Delusion: New Technologies and Expectation Effects · 2005. 7. 27. · 1. The Placebo and Halo Effects • Placebo Effect: a technology has an effect, because the person](https://reader036.vdocument.in/reader036/viewer/2022081623/613efa14c500cf75ab363ab7/html5/thumbnails/14.jpg)
Three Pitfalls to Avoid
![Page 15: Avoiding Self-Delusion: New Technologies and Expectation Effects · 2005. 7. 27. · 1. The Placebo and Halo Effects • Placebo Effect: a technology has an effect, because the person](https://reader036.vdocument.in/reader036/viewer/2022081623/613efa14c500cf75ab363ab7/html5/thumbnails/15.jpg)
1. Equivalent Exchange
• "Maybe it didn't help, but it didn't hurt anything either."
![Page 16: Avoiding Self-Delusion: New Technologies and Expectation Effects · 2005. 7. 27. · 1. The Placebo and Halo Effects • Placebo Effect: a technology has an effect, because the person](https://reader036.vdocument.in/reader036/viewer/2022081623/613efa14c500cf75ab363ab7/html5/thumbnails/16.jpg)
2. Novelty
• "All new tools get this sort of reception at first."
![Page 17: Avoiding Self-Delusion: New Technologies and Expectation Effects · 2005. 7. 27. · 1. The Placebo and Halo Effects • Placebo Effect: a technology has an effect, because the person](https://reader036.vdocument.in/reader036/viewer/2022081623/613efa14c500cf75ab363ab7/html5/thumbnails/17.jpg)
3. Problems Obscured
• "We now know to what degree the results of introducing this tool were influenced by expectation effects, so that's all there is to say about that."
![Page 18: Avoiding Self-Delusion: New Technologies and Expectation Effects · 2005. 7. 27. · 1. The Placebo and Halo Effects • Placebo Effect: a technology has an effect, because the person](https://reader036.vdocument.in/reader036/viewer/2022081623/613efa14c500cf75ab363ab7/html5/thumbnails/18.jpg)
Expectation Effects and the Horizon Report
![Page 19: Avoiding Self-Delusion: New Technologies and Expectation Effects · 2005. 7. 27. · 1. The Placebo and Halo Effects • Placebo Effect: a technology has an effect, because the person](https://reader036.vdocument.in/reader036/viewer/2022081623/613efa14c500cf75ab363ab7/html5/thumbnails/19.jpg)
ExtendedLearning
UbiquitousWireless
IntelligentSearching
EducationalGaming
Social Networks&
Knowledge Webs
Context-Aware Computing
&Augmented
Reality
Classifying Expectation Effects by Likelihood
Time-to-Adoption:One Year or Less
Time-to-Adoption:Two to Three Years
Time-to-Adoption:Four to Five Years
![Page 20: Avoiding Self-Delusion: New Technologies and Expectation Effects · 2005. 7. 27. · 1. The Placebo and Halo Effects • Placebo Effect: a technology has an effect, because the person](https://reader036.vdocument.in/reader036/viewer/2022081623/613efa14c500cf75ab363ab7/html5/thumbnails/20.jpg)
Bibliography•General Overview:
• Draper, S.W. The Hawthorne Effect. URL: http://www.psy.gla.ac.uk/~steve/hawth.html (March 28, 2005)
•Tools for Research:
• Rosenthal, R. and R.L. Rosnow. Essentials of Behavioral Research: Methods and Data Analysis - Second Edition. New York: McGraw-Hill, Inc., 1991. (See esp. Chapter 6)
• Rosenthal, R. Experimenter Effects in Behavioral Research - Enlarged Edition. New York: Irvington Publishers, Inc., 1976. (See esp. Chapters 19-24)
•Experimental Design:
• Rosenthal, R. and L. Jacobson. Pygmalion in the Classroom: Teacher Expectation and Pupils’ Intellectual Development. New York: Holt, Rinehart, and Winston, Inc., 1968. (See esp. Chapter 11)
•Effect Size Index:
• Cohen, J. Statistical Power Analysis for the Behavioral Sciences. New York: Academic Press, 1969. (See esp. Chapter 2)