Slides to accompany Weathington, Cunningham & Pittenger (2010),

Chapter 11: Between-Subjects Designs

1

Objectives

• t-test for independent groups

• Hypothesis testing

• Interpreting t and p

• Statistical power

2

t-test for Independent Groups

• Basic inferential statistic

• Ratio of two measures of variability =

Difference between two group means

Standard Error of the difference between group means

• Allows us to consider effect, relative to error

3

Standard Error of the Difference between Means

4

t-test

• Larger |t-ratio| = greater difference between means

• Based on this we can decide whether to reject Ho

– Usually Ho = µ1 = µ2

• Sampling error may account for some difference, but when t is “large” enough…

5

Hypothesis Testing: t-tests

• Based on estimates of probability

• When α = .05, there is a 5% chance of rejecting Ho when we should not (Type I

error)

– See Figure 11.2 (each tail = 2.5%)

– Region of rejection

• If t falls within the shaded ranges, we reject Ho because probability is so low

6

Figure 11.2

7

Hypothesis Testing Steps

1. State Ho and H1

– Before collecting or examining the data

2. Identify appropriate statistical test(s)

– Based on hypotheses

– Often multiple approaches are possible

– Depends on how well data meet the assumptions of specific statistical tests

8

Hypothesis Testing Steps

3. Set the significance level (α)

– α = p(Type I error)

• Risk of false alarm

• You control

– 1 – α = p(Type II error)

• Risk of miss

• Careful, you might “overcontrol”

9

Hypothesis Testing Steps

4. Determine significance level for t-ratio

– Use appropriate table in Appendix B, df for the test and your selected alpha (α) level to determine tcritical

– If your observed |t ratio| > tcritical reject

Ho

– If your observed p-level is less than α you can also reject Ho

10

Hypothesis Testing Steps

5. Interpreting t-ratio

– Is it statistically significant?

– Is it practically/clinically significant?

• Does the effect size matter, really?

• Book mentions d-statistic

11

Hypothesis Testing Steps5. Interpreting t-ratio

– Magnitude of the effect

• Degree of variance accounted for by the IV

• Omega squared = % of variance accounted for by IV in the DV

– Is there cause and effect?

• Typically requires manipulated IV, randomized assignment, and careful pre- / post- design

12

Correct Interpretation of t and p• If you have a significant t-ratio:

= statistically significant difference between two groups

= IV affects DV

= probability of a Type I error is α

13

Errors in p Interpretation

• Changing α after analyzing the data

– Unethical

– We cannot use p to alter α

• Kills your chances of limiting Type I error risk

• p only estimates the probability of obtaining at least the results you did if the null hypothesis is true, and it is based on sample statistics not fully the case for α

14

Errors in p Interpretation

• Stating that p = odds-against chance

– p = .05 does not mean that the probability of results due to chance was 5% or less

– p is not the probability of committing a Type I error

– Recommended interpretation:

•If p is small enough, I reject the null hypothesis in favor of the alternative hypothesis.

15

Errors in p Interpretation

• Assuming p = probability that H1 is

true (i.e., that the results are “valid”)

– p does not confirm the validity of H1

– Smaller p values do not indicate a more important relationship between IV and DV

•Effect size estimates are required for this

16

Errors in p Interpretation

• Assuming p = probability of replicating results

– The probability of rejecting Ho is not

related to the obtained p-value

• A new statistic, prep is getting some

attention for this purpose (see Killeen, 2005)

17

Statistical Tests & Power

• β = p(Type II error) or p(miss)

• 1 – β = p(correctly rejecting false Ho)

= power

• Four main factors influence statistical power

18

Power: Difference between µ

• Power increases when the difference between µ of two populations is greater

19

Power: Sample Size

• Issue of how well a statistic estimates the population parameter (Fig. 10.5)

• Larger N smaller SEM

• As SEM decreases overlap of sampling distributions for two populations decreases power increases

• Don’t forget about cost20

Power: Variability in Data

• Lots of variability variance in the sampling distribution and greater overlap of two distributions

• Reducing variability reduces SEM overlap decreases power goes up

• Techniques: Use homogeneous samples, reliable measurements

21

Power: α

• Smaller α lower Type I probability lower power

• As p(Type I) decreases, p(Type II) increases (see Figure 11.6)

• As α increases, power increases

– Enlarges the region of rejection

22

23

Estimating Sample Size

• Based on power

• Tables in Appendix B can give you estimates for t-ratios

– Effect size is sub-heading

• Cost / feasibility considerations

• Remember that sample size is not the only influence on statistical power

24

What is Next?

• **instructor to provide details

25