error, bias and objectivity in experimental design anne oxbrougheshare.edgehill.ac.uk/14045/1/mres...
TRANSCRIPT
![Page 1: Error, Bias and Objectivity in Experimental design Anne Oxbrougheshare.edgehill.ac.uk/14045/1/MRes science training Bias and Error.pdf · Systematic error: selection bias & information](https://reader033.vdocument.in/reader033/viewer/2022060305/5f0945ec7e708231d42608c3/html5/thumbnails/1.jpg)
Error, Bias and Objectivity in Experimental design
Anne Oxbrough
![Page 2: Error, Bias and Objectivity in Experimental design Anne Oxbrougheshare.edgehill.ac.uk/14045/1/MRes science training Bias and Error.pdf · Systematic error: selection bias & information](https://reader033.vdocument.in/reader033/viewer/2022060305/5f0945ec7e708231d42608c3/html5/thumbnails/2.jpg)
What does it mean to be objective in science?
• Experimental design should reduce error or bias by:
– Remove pre-conceived ideas
– Avoid bias by poor experimental design
![Page 3: Error, Bias and Objectivity in Experimental design Anne Oxbrougheshare.edgehill.ac.uk/14045/1/MRes science training Bias and Error.pdf · Systematic error: selection bias & information](https://reader033.vdocument.in/reader033/viewer/2022060305/5f0945ec7e708231d42608c3/html5/thumbnails/3.jpg)
Types of error in experimental design
• Random error
– Chance variation in a ‘population’
– Equally misclassifies treatments and controls
• Systematic error
– Misclassifies treatments in one direction and controls in another
– Selection bias
– Confounding variable bias
Good experimental
design
![Page 4: Error, Bias and Objectivity in Experimental design Anne Oxbrougheshare.edgehill.ac.uk/14045/1/MRes science training Bias and Error.pdf · Systematic error: selection bias & information](https://reader033.vdocument.in/reader033/viewer/2022060305/5f0945ec7e708231d42608c3/html5/thumbnails/4.jpg)
https://cirt.gcu.edu/research/developmentresources/research_ready/experimental/error_bias
![Page 5: Error, Bias and Objectivity in Experimental design Anne Oxbrougheshare.edgehill.ac.uk/14045/1/MRes science training Bias and Error.pdf · Systematic error: selection bias & information](https://reader033.vdocument.in/reader033/viewer/2022060305/5f0945ec7e708231d42608c3/html5/thumbnails/5.jpg)
Random Error
• Low precision of outcome
• Outcome is not precise but is true
– Imprecise measuring
– Small sample size
• Decreases with
– Increasing sample size
– Repeating test on different sample of population
![Page 6: Error, Bias and Objectivity in Experimental design Anne Oxbrougheshare.edgehill.ac.uk/14045/1/MRes science training Bias and Error.pdf · Systematic error: selection bias & information](https://reader033.vdocument.in/reader033/viewer/2022060305/5f0945ec7e708231d42608c3/html5/thumbnails/6.jpg)
https://cirt.gcu.edu/research/developmentresources/research_ready/experimental/error_bias
![Page 7: Error, Bias and Objectivity in Experimental design Anne Oxbrougheshare.edgehill.ac.uk/14045/1/MRes science training Bias and Error.pdf · Systematic error: selection bias & information](https://reader033.vdocument.in/reader033/viewer/2022060305/5f0945ec7e708231d42608c3/html5/thumbnails/7.jpg)
Systematic Error
• Low validity of outcome
• Outcome is not true
– Selection bias
– Confounding variable
• Decreases with:
– Knowledge of test system
– Understanding of potential areas of bias
![Page 8: Error, Bias and Objectivity in Experimental design Anne Oxbrougheshare.edgehill.ac.uk/14045/1/MRes science training Bias and Error.pdf · Systematic error: selection bias & information](https://reader033.vdocument.in/reader033/viewer/2022060305/5f0945ec7e708231d42608c3/html5/thumbnails/8.jpg)
https://cirt.gcu.edu/research/developmentresources/research_ready/experimental/error_bias
![Page 9: Error, Bias and Objectivity in Experimental design Anne Oxbrougheshare.edgehill.ac.uk/14045/1/MRes science training Bias and Error.pdf · Systematic error: selection bias & information](https://reader033.vdocument.in/reader033/viewer/2022060305/5f0945ec7e708231d42608c3/html5/thumbnails/9.jpg)
Systematic Error: Confounding variable bias
• The effect of the explanatory variable on the response variable is distorted by the responses association with other factors
![Page 10: Error, Bias and Objectivity in Experimental design Anne Oxbrougheshare.edgehill.ac.uk/14045/1/MRes science training Bias and Error.pdf · Systematic error: selection bias & information](https://reader033.vdocument.in/reader033/viewer/2022060305/5f0945ec7e708231d42608c3/html5/thumbnails/10.jpg)
https://cirt.gcu.edu/research/developmentresources/research_ready/experimental/error_bias
![Page 11: Error, Bias and Objectivity in Experimental design Anne Oxbrougheshare.edgehill.ac.uk/14045/1/MRes science training Bias and Error.pdf · Systematic error: selection bias & information](https://reader033.vdocument.in/reader033/viewer/2022060305/5f0945ec7e708231d42608c3/html5/thumbnails/11.jpg)
https://cirt.gcu.edu/research/developmentresources/research_ready/experimental/error_bias
![Page 12: Error, Bias and Objectivity in Experimental design Anne Oxbrougheshare.edgehill.ac.uk/14045/1/MRes science training Bias and Error.pdf · Systematic error: selection bias & information](https://reader033.vdocument.in/reader033/viewer/2022060305/5f0945ec7e708231d42608c3/html5/thumbnails/12.jpg)
Experimental design: Confounding variables
• Knowledge of study system
• Sound hypothesis
• Control/exclude for confounding variables
• Measure confounding variables
![Page 13: Error, Bias and Objectivity in Experimental design Anne Oxbrougheshare.edgehill.ac.uk/14045/1/MRes science training Bias and Error.pdf · Systematic error: selection bias & information](https://reader033.vdocument.in/reader033/viewer/2022060305/5f0945ec7e708231d42608c3/html5/thumbnails/13.jpg)
Systematic error: selection bias & information bias
• Selection bias:
– Are study subjects similar in all respects apart from explanatory variable?
– Control for as many factors/confounding variables as possible
• Information bias:
– Is information about outcome collected in the same way for all treatments & control?
– Good & rigorous experimental design
![Page 14: Error, Bias and Objectivity in Experimental design Anne Oxbrougheshare.edgehill.ac.uk/14045/1/MRes science training Bias and Error.pdf · Systematic error: selection bias & information](https://reader033.vdocument.in/reader033/viewer/2022060305/5f0945ec7e708231d42608c3/html5/thumbnails/14.jpg)
Avoiding bias
• Knowledge of the system
– Select correct explanatory variables
– Identify cofounding variables
• Random sampling of the population
• Blinding
• Knowledge of potential experimenter bias
– Double blinding
![Page 15: Error, Bias and Objectivity in Experimental design Anne Oxbrougheshare.edgehill.ac.uk/14045/1/MRes science training Bias and Error.pdf · Systematic error: selection bias & information](https://reader033.vdocument.in/reader033/viewer/2022060305/5f0945ec7e708231d42608c3/html5/thumbnails/15.jpg)
What are the implications of unaccounted for error?
• You may conclude something is true when it is not
• You many conclude something is not true when it is
Type II error
Type I error
![Page 16: Error, Bias and Objectivity in Experimental design Anne Oxbrougheshare.edgehill.ac.uk/14045/1/MRes science training Bias and Error.pdf · Systematic error: selection bias & information](https://reader033.vdocument.in/reader033/viewer/2022060305/5f0945ec7e708231d42608c3/html5/thumbnails/16.jpg)
Type I & Type II Errors
• Type I – the probably of rejecting a true null hypothesis
• Type II – the probably of failing to reject a null hypothesis even though it is false
![Page 17: Error, Bias and Objectivity in Experimental design Anne Oxbrougheshare.edgehill.ac.uk/14045/1/MRes science training Bias and Error.pdf · Systematic error: selection bias & information](https://reader033.vdocument.in/reader033/viewer/2022060305/5f0945ec7e708231d42608c3/html5/thumbnails/17.jpg)
Type I & Type II Errors
Treatment & control differ
Treatment & control do not
differ
Conclude treatments do not differ
Conclude treatments do differ
Correct decision
Type 1 errorProbability = alpha
Aim to lower alphaRepeat study
Robust experiment
Type II errorProbability = beta
Correct decision
Truth
Experim
ent
Large sample sizeHypothesise large differences
![Page 18: Error, Bias and Objectivity in Experimental design Anne Oxbrougheshare.edgehill.ac.uk/14045/1/MRes science training Bias and Error.pdf · Systematic error: selection bias & information](https://reader033.vdocument.in/reader033/viewer/2022060305/5f0945ec7e708231d42608c3/html5/thumbnails/18.jpg)
Bias may occur at any stage
• Literature review
• Experimental design
• Data collection
• Analysis
• Interpretation of results & conclusions
• Publication
![Page 19: Error, Bias and Objectivity in Experimental design Anne Oxbrougheshare.edgehill.ac.uk/14045/1/MRes science training Bias and Error.pdf · Systematic error: selection bias & information](https://reader033.vdocument.in/reader033/viewer/2022060305/5f0945ec7e708231d42608c3/html5/thumbnails/19.jpg)
Group work – bias & error
Design a simple, practical scientific experiment
− Set hypothesis/hypotheses
− Choose dependent and independent variables
− Construct experimental and control groups
− Consider sampling approach
− Highlight sources of bias at all stages of research
In pairs
Choose a topic from the list of examples (some more workable than others!), or devise your own
30 minutes preparation time
2-3 minute presentation (PowerPoint if preparation time permits)
Example topics
1. Alder trees thrive in waterlogged conditions
2. Dogs only see in black and white
3. Listening to classical music increases intelligence
4. Women are better multi-taskers than men
5. Driving on the left is safer than driving on the right
6. Red smarties taste best
7. Elvis is alive
![Page 20: Error, Bias and Objectivity in Experimental design Anne Oxbrougheshare.edgehill.ac.uk/14045/1/MRes science training Bias and Error.pdf · Systematic error: selection bias & information](https://reader033.vdocument.in/reader033/viewer/2022060305/5f0945ec7e708231d42608c3/html5/thumbnails/20.jpg)
Supervisor discussion topics
• For your broader discipline and your specific project:
– Identify sources of random and systematic bias
– Identify ways to reduce these
– Identify implications if you don’t reduce error
• Academic
• Wider Impact
• Consider & identify examples of these for every stage of the research process
![Page 21: Error, Bias and Objectivity in Experimental design Anne Oxbrougheshare.edgehill.ac.uk/14045/1/MRes science training Bias and Error.pdf · Systematic error: selection bias & information](https://reader033.vdocument.in/reader033/viewer/2022060305/5f0945ec7e708231d42608c3/html5/thumbnails/21.jpg)
Resources & Acknowledgements
• https://cirt.gcu.edu/research/developmentresources/research_ready/experimental/error_bias
• https://courses.lumenlearning.com/boundless-psychology/chapter/bias-in-psychological-research/
• https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2917255/
• https://www.thoughtco.com/difference-between-type-i-and-type-ii-errors-3126414