PTP 560

• Research Methods

Week 9

Thomas Ruediger, PT

Calculate Sn, Sp, LRs

Confidence Intervals with small samples

What is small sample size?Less than 30 (one of those special numbers in stats)Sampling distribution tends to spread outStandard normal curve not adequate

Use the t-distributionsTheoretical sampling distributionsFlatter peak, wider at the tailsApproaches normal curve as sample size increasesUse values of t instead of zDescribed by degrees of freedom (n-1 for confidence

intervals)

Hypothesis testing• Are the differences– Representative of “real” effects?– Just by chance?

• Null hypothesis ( HO)– Means are not different– Stated in terms of population parameter– μA=μB

• Alternative hypothesis (H1)– Difference too large to be by chance– μA≠μB

– May be directional or non-directional

TruthD

ecis

ion

Ho is True

Do Not Reject Ho

α

Correct

β

Reject Ho

Ho is False

CorrectType I error

Type II error

Type I Error• Significance Level (Alpha level , α level)• Your choice of how much risk you are willing to

take of saying there is a difference when there really is no difference

• Set this before the study– Conventionally is 0.05• This is arbitrary, but almost always what we choose

– Choose the level based on the Type I error concern

Type I Error

• Probability Values (Evaluated after the study)– Probability of finding this big a difference by chance

• p = .07 of this big a difference by chance

– You are not stating the probability of the inverse• p = .93 that it is real difference is not appropriate

– Compare p-value to alpha level if greater

– Compare your p-value (calculated after the study) with your α level (set before the study)

– If the p-value is less than α, reject the null– If the p-value is greater than α, fail to reject the null

Type II ErrorStatistical Power

• Beta (β)– Probability of failing to reject a false HO(null hypotheses)

– Β of 0.20 is 20% chance we will make a Type II error• Statistical Power– Complement of β (not compliment)– In this example 0.80 (1.00 – 0.20 = 0.80)– 80% probability of correctly rejecting the null

• Before: a priori - power used to determine sample size• After: post hoc – power reported if HO not rejected

“If there was a difference, could we have found it?”

Determinants ofStatistical Power

• Significance Criterion– As α increases, power increases(As α increases from 0.05 to 0.10, power increases)

• Variance– As variance decreases, power increases

• Sample size– As sample size increases, power increases

• Effect size (difference b/w the group means)

– As effect size increases, power increases

z-• z - score represents the distance between:– A sample score and– Sample mean– Divided by the standard deviation

You will see this in osteoporosis scores(+2 for z- score is 2 SD away from a healthily woman mean)

• z - ratio represents the distance between:– A sample mean and– Population mean– Divided by the standard error of the mean

Critical Region That portion of the curve above and below z

If calculated z > critical z, reject HO

One or two tailed test? Non directional hypothesis– two tailed

z of 2.00 encompasses 95.44 % (non-critical) 4.66% is the critical area

z of 1.96 encompass 95%, Critical region is 5%, 2.5% in each tail of a non-directional test

Directional hypothesis– one tailed z of 1.645 encompasses 95%

Critical region is 5%, all in one tail of a directional test, while NON-direction will be 2.5% Practically, you are disregarding everything in the other tail

Table A.1 back of P & W

Figure A: Intervention 1 is different than Intervention 2

Figure B: Intervention 1 is less effective than Intervention 2

Parametric Statistics• Used to estimate population parameters• Based on assumptions– Randomly drawn from a normally distributed population– Variances in the samples equal (at least roughly)– Interval or ratio scale

• Classically, if assumptions violated, use non-parametric tests

• Many view parametric stats as Robust enough to withstand even major violation

t-test Examines two means

Two groups Two conditions/two performances

Statistical significance based onDifference in the means

Between the groups The effect size

Variance Within the groups How variable are the scores

Fig 19.1

t-test Based on a ratio

Difference between group means/Variability within groups

Difference between means Treatment effect and error variance

One mean- second mean and variability between the groups. In both the numerator and denominator of t-test ratio, so holds it to zero.

Variability within groups Error variance alone

Equal and unequal variances affect t-ratio SPSS and most other packages automatically test for this. Where?

Ratio can be written: Treatment effect and error variance/Error variance

NOTE:Error variance Not mistakes Is anything that is not due to the independent variable

t-test If the null is true

Ratio reduces to: Error/Error

The bigger the difference - The bigger the ratioHow does the ratio get bigger?

This ratio is compared to the critical Determines significance

Is the ratio significant? Based on critical value (but now t instead of z) Entering arguments (Table A.2)

Alpha level (almost always 0.05 Degrees of freedom ( one or a few less than n )

CI can be constructed for where the true mean difference lies

t-test Independent t-test

Usually random assignmentCan be convenience assignmentNo inherent relationship between the groupsDegrees of freedom = total sample size – 2

Paired t-testAn inherent relationship between the groups

Self (repeated measure test) Twins

Difference scores for each pair comparedDegrees of freedom = number of paired scores – 1

Find this in P & W or in an SPSS output table – don’t calculate for this class!

ANOVA Examines three (or more) means

Three (or more) groups Three (or more) conditions/ three (or more) performances

Statistical significance based on Difference in the means

Between the groups The effect size

Variance Within the groups How variable are the scores

This should sound familiar

ANOVA Based on ratio

Treatment effect and error variance/Error variance

For ANOVA it is the F-ratio Derived from the Sum of Squares (SS) Larger the SS the larger the ______________? Calculate SS

(each score minus sample mean, square each result, sum them)

Then determine the Mean Square (MS) MSb = SSb/dfb (dfb = one less than the number of groups) MSe = SSe/df e (df e = total N – number of groups)

F statistic is the ratio = MSb/Mse

Ratio of the between groups variance to error variance

Find this in P & W or in an SPSS output table – don’t calculate for this class!