confidence intervals, effect size and power
DESCRIPTION
Confidence Intervals, Effect Size and Power. Chapter 8. Hyde, Fennema & Lamon (1990). Mean gender differences in mathematical reasoning were very small. When extreme tails of the distributions were removed, differences were smaller and reversed direction (favored girls) - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Confidence Intervals, Effect Size and Power](https://reader035.vdocument.in/reader035/viewer/2022062218/568161c5550346895dd1aa23/html5/thumbnails/1.jpg)
Confidence Intervals, Effect Size and Power
Chapter 8
![Page 2: Confidence Intervals, Effect Size and Power](https://reader035.vdocument.in/reader035/viewer/2022062218/568161c5550346895dd1aa23/html5/thumbnails/2.jpg)
Hyde, Fennema & Lamon (1990)
> Mean gender differences in mathematical reasoning were very small.
> When extreme tails of the distributions were removed, differences were smaller and reversed direction (favored girls)
> The gender differences depended on task• Females were better with computation.• Males were better with problem solving.
![Page 3: Confidence Intervals, Effect Size and Power](https://reader035.vdocument.in/reader035/viewer/2022062218/568161c5550346895dd1aa23/html5/thumbnails/3.jpg)
Men, Women, and Math
> An accurate understanding of gender differences may not be found in “significant” effects.
> Each distribution has variability.> Each distribution has overlap.
![Page 4: Confidence Intervals, Effect Size and Power](https://reader035.vdocument.in/reader035/viewer/2022062218/568161c5550346895dd1aa23/html5/thumbnails/4.jpg)
A Gender Difference in Mathematics Performance – amount of overlap as
reported by Hyde (1990)
![Page 5: Confidence Intervals, Effect Size and Power](https://reader035.vdocument.in/reader035/viewer/2022062218/568161c5550346895dd1aa23/html5/thumbnails/5.jpg)
Confidence Intervals
> Point estimate: summary statistic – one number as an estimate of the population• e.g., mean
> Interval estimate: based on our sample statistic, range of sample statistics we would expect if we repeatedly sampled from the same population• e.g., Confidence interval
![Page 6: Confidence Intervals, Effect Size and Power](https://reader035.vdocument.in/reader035/viewer/2022062218/568161c5550346895dd1aa23/html5/thumbnails/6.jpg)
> Interval estimate that includes the mean we would expect for the sample statistic a certain percentage of the time were we to sample from the same population repeatedly• Typically set at 95%
> The range around the mean when we add and subtract a margin of error
> Confirms findings of hypothesis testing and adds more detail
Confidence Intervals
![Page 7: Confidence Intervals, Effect Size and Power](https://reader035.vdocument.in/reader035/viewer/2022062218/568161c5550346895dd1aa23/html5/thumbnails/7.jpg)
Calculating Confidence Intervals
> Step 1: Draw a picture of a distribution that will include confidence intervals
> Step 2: Indicate the bounds of the CI on the drawing
> Step 3: Determine the z statistics that fall at each line marking the middle 95%
> Step 4: Turn the z statistic back into raw means
> Step 5: Check that the CIs make sense
![Page 8: Confidence Intervals, Effect Size and Power](https://reader035.vdocument.in/reader035/viewer/2022062218/568161c5550346895dd1aa23/html5/thumbnails/8.jpg)
Confidence Interval for z-test
Mlower = - z(σM) + Msample
Mupper = z(σM) + Msample
> Think about it: Why do we need two calculations?
![Page 9: Confidence Intervals, Effect Size and Power](https://reader035.vdocument.in/reader035/viewer/2022062218/568161c5550346895dd1aa23/html5/thumbnails/9.jpg)
A 95% Confidence Interval, Part I
![Page 10: Confidence Intervals, Effect Size and Power](https://reader035.vdocument.in/reader035/viewer/2022062218/568161c5550346895dd1aa23/html5/thumbnails/10.jpg)
A 95% Confidence Interval, Part II
![Page 11: Confidence Intervals, Effect Size and Power](https://reader035.vdocument.in/reader035/viewer/2022062218/568161c5550346895dd1aa23/html5/thumbnails/11.jpg)
A 95% Confidence Interval, Part III
![Page 12: Confidence Intervals, Effect Size and Power](https://reader035.vdocument.in/reader035/viewer/2022062218/568161c5550346895dd1aa23/html5/thumbnails/12.jpg)
> Which do you think more accurately represents the data: point estimate or interval estimate?• Why?
> What are the advantages and disadvantages of each method?
Think About It
![Page 13: Confidence Intervals, Effect Size and Power](https://reader035.vdocument.in/reader035/viewer/2022062218/568161c5550346895dd1aa23/html5/thumbnails/13.jpg)
Effect Size: Just How Big Is the Difference?
> Significant = Big?> Significant = Different?> Increasing sample size will make us
more likely to find a statistically significant effect
![Page 14: Confidence Intervals, Effect Size and Power](https://reader035.vdocument.in/reader035/viewer/2022062218/568161c5550346895dd1aa23/html5/thumbnails/14.jpg)
What is effect size?
> Size of a difference that is unaffected by sample size
> Standardization across studies
![Page 15: Confidence Intervals, Effect Size and Power](https://reader035.vdocument.in/reader035/viewer/2022062218/568161c5550346895dd1aa23/html5/thumbnails/15.jpg)
Effect Size & Mean Differences Imagine both represent significant
effects: Which effect is bigger?
![Page 16: Confidence Intervals, Effect Size and Power](https://reader035.vdocument.in/reader035/viewer/2022062218/568161c5550346895dd1aa23/html5/thumbnails/16.jpg)
Effect Size and Standard DeviationNote the spread in the two figures:
Which effect size is bigger?
![Page 17: Confidence Intervals, Effect Size and Power](https://reader035.vdocument.in/reader035/viewer/2022062218/568161c5550346895dd1aa23/html5/thumbnails/17.jpg)
> Decrease the amount of overlap between two distributions:
1. Their means are farther apart 2. The variation within each population is
smaller
Increasing Effect Size
![Page 18: Confidence Intervals, Effect Size and Power](https://reader035.vdocument.in/reader035/viewer/2022062218/568161c5550346895dd1aa23/html5/thumbnails/18.jpg)
> Cohen’s d Estimates effect size> Assesses difference between means
using standard deviation instead of standard error
Md
Calculating Effect Size
![Page 19: Confidence Intervals, Effect Size and Power](https://reader035.vdocument.in/reader035/viewer/2022062218/568161c5550346895dd1aa23/html5/thumbnails/19.jpg)
![Page 20: Confidence Intervals, Effect Size and Power](https://reader035.vdocument.in/reader035/viewer/2022062218/568161c5550346895dd1aa23/html5/thumbnails/20.jpg)
Prep
> Probability of replicating an effect given a particular population and sample size
> To Calculate• Step 1: Calculate the p value associated with
our statistic (in Appendix B)• Step 2: Use the following equation in Excel
> =NORMSDIST(NORMSINV(1-P)/(SQRT(2)))– Use the actual p value instead of P in the equation
![Page 21: Confidence Intervals, Effect Size and Power](https://reader035.vdocument.in/reader035/viewer/2022062218/568161c5550346895dd1aa23/html5/thumbnails/21.jpg)
Statistical Power
> The measure of our ability to reject the null hypothesis, given that the null is false• The probability that we will reject the null
when we should• The probability that we will avoid a Type II
error
![Page 22: Confidence Intervals, Effect Size and Power](https://reader035.vdocument.in/reader035/viewer/2022062218/568161c5550346895dd1aa23/html5/thumbnails/22.jpg)
Calculating Power
> Step 1: Determine the information needed to calculate power• Population mean, population standard
deviation, sample mean, samp0le size, standard error (based on sample size)
> Step 2: Determine a critical z value and raw mean, to calculate power
> Step 3: Calculate power: the percentage of the distribution of the means for population 2 that falls above the critical value
![Page 23: Confidence Intervals, Effect Size and Power](https://reader035.vdocument.in/reader035/viewer/2022062218/568161c5550346895dd1aa23/html5/thumbnails/23.jpg)
![Page 24: Confidence Intervals, Effect Size and Power](https://reader035.vdocument.in/reader035/viewer/2022062218/568161c5550346895dd1aa23/html5/thumbnails/24.jpg)
Statistical Power: The Whole Picture
![Page 25: Confidence Intervals, Effect Size and Power](https://reader035.vdocument.in/reader035/viewer/2022062218/568161c5550346895dd1aa23/html5/thumbnails/25.jpg)
Increasing Alpha
![Page 26: Confidence Intervals, Effect Size and Power](https://reader035.vdocument.in/reader035/viewer/2022062218/568161c5550346895dd1aa23/html5/thumbnails/26.jpg)
Two-Tailed Verses One-Tailed Tests
![Page 27: Confidence Intervals, Effect Size and Power](https://reader035.vdocument.in/reader035/viewer/2022062218/568161c5550346895dd1aa23/html5/thumbnails/27.jpg)
Increasing Sample Size or Decreasing Standard Deviation
![Page 28: Confidence Intervals, Effect Size and Power](https://reader035.vdocument.in/reader035/viewer/2022062218/568161c5550346895dd1aa23/html5/thumbnails/28.jpg)
Increasing the Difference between the Means
![Page 29: Confidence Intervals, Effect Size and Power](https://reader035.vdocument.in/reader035/viewer/2022062218/568161c5550346895dd1aa23/html5/thumbnails/29.jpg)
> Larger sample size increases power> Alpha level
• Higher level increases power (e.g., from .05 to .10)
> One-tailed tests have more power than two-tailed tests
> Decrease standard deviation> Increase difference between the means
Factors Affecting Power
![Page 30: Confidence Intervals, Effect Size and Power](https://reader035.vdocument.in/reader035/viewer/2022062218/568161c5550346895dd1aa23/html5/thumbnails/30.jpg)
Meta-Analysis
> Meta-analysis considers many studies simultaneously.
> Allows us to think of each individual study as just one data point in a larger study.
![Page 31: Confidence Intervals, Effect Size and Power](https://reader035.vdocument.in/reader035/viewer/2022062218/568161c5550346895dd1aa23/html5/thumbnails/31.jpg)
The Logic of Meta-Analysis
> STEP 1: Select the topic of interest> STEP 2: Locate every study that has> been conducted and meets the criteria> STEP 3: Calculate an effect size, often
Cohen’s d, for every study> STEP 4: Calculate statistics and create
appropriate graphs
![Page 32: Confidence Intervals, Effect Size and Power](https://reader035.vdocument.in/reader035/viewer/2022062218/568161c5550346895dd1aa23/html5/thumbnails/32.jpg)
Meta-Analysis and ElectronicDatabases
![Page 33: Confidence Intervals, Effect Size and Power](https://reader035.vdocument.in/reader035/viewer/2022062218/568161c5550346895dd1aa23/html5/thumbnails/33.jpg)
> Why would it be important to know the statistical power of your study?
Think About It