Chapter 15

Analysis and Reporting of Quantitative Data

Inferential Analysis

• Enables inference of results from a carefully selected sample to an overall population

• Quantifies the potential effects of error on the results

General Rules for Quantitative Analysis

• Statistical tests are selected a priori– The acceptable level of significance is also

selected before analysis begins

• Run all of the identified tests

• Report all of the tests that were run– Selective reporting is a source of bias

Type of Analysis Driven By…

• The goals of the analysis

• Assumptions of the data

• Number of variables in the analysis

Statistical Inference

• Intended to answer two fundamental questions:– How probable is it that the differences

between observed results and those expected on the basis of the null hypothesis have been produced by chance alone?

– How reliable are the results obtained?

Statistical Inference

• Questions about reliability answered by setting confidence limits

• Questions about probability answered by hypothesis testing

• Conclusions concerned with probability of drawing an erroneous conclusion

Estimation

• Point estimate: this sample statistic equals the population parameter

• Interval estimate: the range of numbers we believe will include the population parameter

Statistical estimation allows for determination of the amount of uncertainty in the estimate

Confidence Interval

Two numerical values defining an interval that we believe, with an identified level of confidence,

actually includes the estimated population parameter

Calculation of Confidence Interval for the Mean

• Determination of an acceptable confidence level a priori (1- )

• Identification of coefficient Z

• Calculation of standard error (s / n)

• Determination of the mean

Hypothesis Testing

• Statistical test of significance– Between a sample and a known population – Between two samples– Between two variables in a sample

• Can be used to test differences between:– Means– Proportions– Variances

Inferential Statistics

• If a difference is detected, may be due to:– Experimental treatment caused the effect– Sampling error caused the effect (chance)

• We cannot prove the experiment caused the difference

• We can estimate the probability it was caused by error

Steps in Hypothesis Testing

1. State research question as a statistical hypothesis

2. Select the level of significance for the statistical test (alpha)

3. Decide on the appropriate test statistic

4. Determine the value the test statistic must attain to be declared significant (critical value)

5. Perform the calculations for the test statistic

6. Apply decision rule and draw conclusions

Hypotheses

• H0: The null hypothesis

– e.g. H0: = 30

• HA: The alternative hypothesis

– e.g. HA: 30

The null hypothesis is rejected only if data presents sufficiently strong evidence to support the alternative

Steps in Hypothesis Testing

1. State research question as a statistical hypothesis

2. Select the level of significance for the statistical test (alpha)

3. Decide on the appropriate test statistic

4. Determine the value the test statistic must attain to be declared significant (critical value)

5. Perform the calculations for the test statistic

6. Apply decision rule and draw conclusions

Level of Significance

• Level of Significance:– Preset standard that is considered significant

in determining differences– Compare to p value: the probability of a type I

error that the researcher is willing to risk– Most common: .05 and .01

• Set a priori (alpha)

Steps in Hypothesis Testing

1. State research question as a statistical hypothesis

2. Select the level of significance for the statistical test (alpha)

3. Decide on the appropriate test statistic

4. Determine the value the test statistic must attain to be declared significant (critical value)

5. Perform the calculations for the test statistic

6. Apply decision rule and draw conclusions

Most Inferential Tests...

Based upon difference between parameter estimates in the sample and the population or between two samples

Difference between n and N estimates

Standard error

Steps in Hypothesis Testing1. State research question as a statistical

hypothesis

2. Select the level of significance for the statistical test (alpha)

3. Decide on the appropriate test statistic

4. Determine the value the test statistic must attain to be declared significant (critical value)

5. Perform the calculations for the test statistic

6. Apply decision rule and draw conclusions

Distributions for Probability

• Each parameter has its own distribution– Variance: F– Proportion: 2

– Mean: t– Correlation: t

• Each finite sample will have its own unique probability distribution

• Closer to normality as sample size increases

Steps in Hypothesis Testing

1. State research question as a statistical hypothesis

2. Select the level of significance for the statistical test (alpha)

3. Decide on the appropriate test statistic

4. Determine the value the test statistic must attain to be declared significant (critical value)

5. Perform the calculations for the test statistic

6. Apply decision rule and draw conclusions

Hypothesis test of means

• One sample tests of means

• H0: x =

• HA: x

• Calculate test statistic: t = x - SE (x)

• Compare to critical value from t distribution• Apply decision rule to reject / do not reject the

null hypothesis

Steps in Hypothesis Testing1. State research question as a statistical

hypothesis

2. Select the level of significance for the statistical test (alpha)

3. Decide on the appropriate test statistic

4. Determine the value the test statistic must attain to be declared significant (critical value)

5. Perform the calculations for the test statistic

6. Apply decision rule and draw conclusions

Decision Rules

• All possible values of the test statistic are divided into 2 regions

• Rejection region– Those values of the test statistic with a low

probability of occurrence if H0 is true

– Statistically constructed at 5%– Critical value: cut point for the rejection

region

Possible Decisions

• A significant difference is detected between the actual values and the hypothesized values:– Reject the H0

• A significant difference is not detected between the actual values and the hypothesized values– Do not reject the HA

Type I and II Errors

Null Hypothesisis True

Null Hypothesisis False

Reject NullHypothesis

Type I ErrorRisk =

Correct

Do Not RejectNull

Hypothesis

Correct Type II ErrorRisk =

Type I and II Errors

• Type I Error: Treatment doesn’t work, but we think it does– Leads to unwarranted change– Control by appropriately setting

• Type II Error: Treatment works, but we think it doesn’t– Leads to missed opportunities– Designated as , controlled by sampling

The t Test of Means

• Differences between mean values:– Between a sample and a known population

value• One sample t

– Between two independent samples• Independent samples t test

– Between two time periods for the same group• Paired samples t test

The Chi Square test

• Tests for differences in rates, proportions, or probabilities– Between a sample proportion and a known

population proportion• Chi square test of model fit

– Between two independent samples• Chi square test of independence

– Between two variables in a single sample• Chi square test of association

Analysis of Variance

• Tests for differences in means in more than two groups– Single dependent variable

• ANOVA

– Single dependent variable with potential covariates

• ANCOVA

– Single dependent variable measured over more than two time periods

• Repeated Measures ANOVA

Most Common Reported Statistics

• Descriptive statistics about sample and variables

• Analysis of group equivalency

• Statistics about the role of error

• Statistics to evaluate magnitude of effect

• Statistics to determine confidence level