![Page 1: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/1.jpg)
How to read
academic research
(even if you’re not an expert)
Dr. Russell James III, Texas Tech University www.EncourageGenerosity.com
![Page 2: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/2.jpg)
Rule 1
Don’t Freak Out!
![Page 3: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/3.jpg)
You don’t need to eat the whole cow!
You can get important
concepts out of a research
article without fully
understanding every detail
![Page 4: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/4.jpg)
How do you eat a cake with rocks in it?
Don’t try to eat the rocks
![Page 5: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/5.jpg)
Questions for an article
1.Do I care about the research topic?
2.Do I believe the findings?
3.So what?
![Page 6: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/6.jpg)
Abstract: Do I care?Tables: What did they really find?Methods: Do I believe the table?Discussion: So what?Lit. Review: What did we already know?
![Page 7: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/7.jpg)
Title and Abstract: Do I care?
![Page 8: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/8.jpg)
Tables: What did they find?
![Page 9: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/9.jpg)
Methods: Should I believe
the table?
![Page 10: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/10.jpg)
Discussion: So What?
![Page 11: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/11.jpg)
Literature Review:What did we
already know?
![Page 12: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/12.jpg)
Should you believe the findings?Research is messy. Research often disagrees. We want to be able to distinguish strong results from weak ones.
![Page 13: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/13.jpg)
Bad news
Knowing whether you should believe the findings usually requires some statistics
![Page 14: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/14.jpg)
Core statistics concepts you must know
1. Association v. Causation2. Correlation v. Multiple Regression3. Significance v. Magnitude
![Page 15: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/15.jpg)
Association v. Causation
Association: A & B tend to occur together more frequently than one would expect by random chance
![Page 16: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/16.jpg)
Explaining Associations1. Random chance (stuff happens)2. A causes B (sometimes)3. B causes A (sometimes)4. Something else causes both A & B
(sometimes)
![Page 17: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/17.jpg)
Sleeping in your shoes is associated with waking up with a headache.
Why?
![Page 18: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/18.jpg)
1. Random chance2. Sleeping in shoes causes headaches3. The very early stages of a forthcoming
headache causes sleeping in shoes4. Going to bed drunk causes both results
![Page 19: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/19.jpg)
Association v. Causation
• Statistics can show only association
• Statistics can NEVER show causation
We infer causation from experimental design or theory combined with statistical association
![Page 20: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/20.jpg)
Explaining associations:1. Random chance2. A causes B3. B causes A4. Something else causes both A & B
Statistics can easily determine
this
less so with these
![Page 21: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/21.jpg)
Correlationv.
Multiple Regression
![Page 22: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/22.jpg)
Multiple Regression: Above is true when comparing those otherwise similar in certain ways
Correlation: A & B tend to occur together more frequently than one would expect by random chance
![Page 23: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/23.jpg)
CorrelationHigher education and charitable giving tend to occur together (more frequently than one would expect by random chance)
![Page 24: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/24.jpg)
Multiple RegressionHigher education and charitable giving tend to occur together (more frequently than one would expect by random chance)comparing those with otherwise similar income and wealth.
![Page 25: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/25.jpg)
Explaining Associations:1. Random chance2. A causes B3. B causes A4. Something else
causes both A & B
Multiple regression allows us to exclude specific items from #4, unless we can’t or didn’t measure it.
![Page 26: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/26.jpg)
G.E. Quinn, C.H. Shin, M. Maquire, R. Stone (University of Pennsylvania Medical School), 1999, Myopia and Ambient Lighting at Night, Nature, 399, 113.
Nature says kids’ nightlights cause myopia
“Although it does not establish a causal link, the statistical strength of the association of night-time light exposure and childhood myopia does suggest that the absence of a daily period of darkness during early childhood is a potential precipitating factor in the development of myopia.”
![Page 27: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/27.jpg)
G.E. Quinn, C.H. Shin, M. Maquire, R. Stone (University of Pennsylvania Medical School), 1999, Myopia and Ambient Lighting at Night, Nature, 399, 113.
Nature says kids’ nightlights cause myopia
1. Random chance
2. A causes B
3. B causes A
4. Something else causes both A & B
![Page 28: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/28.jpg)
J. Gwiazda, E. Ong, R. Held, F. Thorn (New England College of Optometry), 2000, Myopia and Ambient Night-Time Lighting, Nature, 399, 113.
Rebuttal: Maybe parents’ myopia causes both nightlights and child’s myopia?
“…we find that myopic parents are more likely to employ night-time lighting aids for their children. Moreover, there is an association between myopia in parents and their children…”
“…Quinn et al.’s study should have controlled for parental myopia.”
![Page 29: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/29.jpg)
Significance v.
Magnitude
![Page 30: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/30.jpg)
Statistics tests a small sample to predict the whole population
Significance shows how likely our result might have been due to an unusual random sample, rather than an actual difference in the population
![Page 31: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/31.jpg)
Most papers report some measure of statistical significance (chance that the association was due to a weird random sample)
• p-value• confidence interval
![Page 32: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/32.jpg)
How likely is it to randomly draw these five fruits from a truckload with as many apples as oranges?
p-value
![Page 33: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/33.jpg)
p-value
p<.05 = there is less than a 5% chance that the result was caused by an unusual random sample where there was no actual (population) difference
![Page 34: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/34.jpg)
Was there a significant gender difference in planned givers with a will v. a trust?
No
![Page 35: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/35.jpg)
![Page 36: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/36.jpg)
This (sample) difference could have easily occurred even if the two (population) groups were the same
![Page 37: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/37.jpg)
It DOES NOT mean the two (population) groups do not differ, only that WE CAN’T TELL.
![Page 38: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/38.jpg)
No “*” means we can’t confidently tell the effect of this item
![Page 39: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/39.jpg)
95% Confidence intervalIf you kept taking random samples, 95% of the time the true (population) value would appear inside the confidence interval associated with each sample
PopulationAverage Strength
SampleAverage Strength
Confidence Interval
![Page 40: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/40.jpg)
S. Huck and I. Rasul (2008) Testing consumer theory in the field: Private consumption versus charitable goods
Dashed line is a 95% confidence interval
![Page 41: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/41.jpg)
How likely is it to randomly draw these five fruits from a truckload with as many apples as oranges?
Would your answer change if I got to draw 20 times to find this group?
Multiple Comparisons Problem
![Page 42: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/42.jpg)
If all variables are random, about one out of 20 will have a p-value<.05
![Page 43: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/43.jpg)
“We tested 100 items and found 5 to be significant at p<.05.”
![Page 44: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/44.jpg)
Significance v. Magnitude
It is possible to be highly confident of a very small effect. This may be publishable, but not practically important.
![Page 45: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/45.jpg)
Numbers (coefficients) resulting
from complex statistical techniques may not be directly
interpretable in terms of real world magnitude
![Page 46: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/46.jpg)
The impact of children
on the probability
of exclusively
secular giving is
“-0.089”, but the meaning
of that number is not easily translated
![Page 47: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/47.jpg)
Even with complex techniques, we can easily compare sign and
magnitude relative to other variables
![Page 48: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/48.jpg)
Race and education factors are
3-4 times as large.
More children have an opposite
relationship compared with more education.
![Page 49: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/49.jpg)
Odds ratios are differentUsually you can compare sign and size, but odds ratios are always positive
![Page 50: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/50.jpg)
Odds ratios: the odds of an event occurring in one group over the odds of it occurring
in another group <1 negative; >1 positive; =1 none
![Page 51: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/51.jpg)
Pamala Weipking (2008) Giving to particular charitable organizations: Do materialists support local organizations and do Democrats donate to animal protection?
Odds ratios <1 correspond with negative coefficient numbers in other reporting
![Page 52: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/52.jpg)
Finding academic research articles
Includes everything, even working papers and industry literature
ISI ranked academic journals articles only
![Page 53: How to read academic research (beginner's guide)](https://reader034.vdocument.in/reader034/viewer/2022042607/54c6f5b84a795929038b4572/html5/thumbnails/53.jpg)
How to read
academic research
(even if you’re not an expert)
Dr. Russell James III, Texas Tech University www.EncourageGenerosity.com