educational research: data analysis and interpretation – 2 inferential statistics edu 8603...
TRANSCRIPT
Educational Research:Educational Research:Data analysis and interpretation – 2Data analysis and interpretation – 2Inferential statisticsInferential statistics
EDU 8603
Educational Research
Richard M. Jacobs, OSA, Ph.D.
Statistics...Statistics...
A set of mathematical procedures for describing, synthesizing, analyzing, and interpreting quantitative data…the selection of an appropriate statistical
technique is determined by the research design, hypothesis, and the data collected
inferential statisticsinferential statistics......…mathematical tools that permit the
researcher to generalize to a population of individuals based upon information obtained from a limited number of research participants
sampling errorsampling error......…the differences in samples due to
random fluctuations within the population
…sampling errors vary in size …but are normally distributed around
the population mean () …and take the shape of a bell curve
standard errorstandard error......…the standard deviation of the sample
means (SEx)
…tells the researcher by how much the researcher would expect the sample means to differ if the researcher used other samples from the same population
but......…the researcher does not have to
select a large number of samples from a population to estimate the standard error
a mathematical formula can be used to estimate the standard error......
SD .
SEx = √ N - 1
…a smaller standard error indicates less sampling error
…the major factor affecting the size of the standard error of the mean is sample size
…but, the size of the population standard deviation also affects the standard error of the mean
The null hypothesis (HThe null hypothesis (H00)...)...
the statement that the difference between two sample means is due to random, chance, sampling error…indicates that there is no true
difference or relationship between parameters in the populations
the null hypothesis differs in most instances from the research research hypothesis hypothesis (H1)…which states that one method is
expected to be more effective than another
rejecting the null hypothesis provides evidence (but not proof) that the treatment had an effect…in other words, that the difference between
dependent variables is due to something other than random, chance, sampling error
The research question, then, is:…whether to accept the null
hypothesis or to reject it
There are four possibilities:
1. The null hypothesis is true and the researcher concludes that it is true
A = B…a correct decision
2. The null hypothesis is false and the researcher concludes that it is false
A ≠ B…a correct decision
3. The null hypothesis is true but the researcher concludes that it is false
A = B…an incorrect decision
4. The null hypothesis is false but the researcher concludes that it is true
A ≠ B…an incorrect decision
Decisions concerning rejecting the Decisions concerning rejecting the null hypothesis…null hypothesis…
The true status of the null hypothesis…
True False
The
res
earc
her’s
dec
isio
n ab
out
the
null
hypo
thes
is…
True
False
Correct Incorrect
Incorrect Correct
Decisions concerning rejecting the Decisions concerning rejecting the null hypothesis…null hypothesis…
The true status of the null hypothesis…
True False
The
res
earc
her’s
dec
isio
n ab
out
the
null
hypo
thes
is…
True
False
CorrectType II Error
Type I Error
Correct
researchers use a test of significancetest of significance to determine whether to reject or fail to reject the null hypothesis…involves pre-selecting a level of probability, “α”
(e.g., α = .05) that serves as the criterion to determine whether to reject or fail to reject the null hypothesis
Steps in using inferential statistics…Steps in using inferential statistics…
1. select the test of significance2. determine whether significance test will be two-tailed or one tailed
3. select α (alpha), the probability level
4. compute the test of significance5. consult table to determine the significance of the results
Tests of significance...Tests of significance...
statistical formulas that enable the researcher to determine if there was a real difference between the sample means
…different tests of significance account for different factors including: the scale of measurement represented by the data; method of participant selection, number of groups being compared, and, the number of independent variables
…the researcher must first decide whether a parametricparametric or nonparametricnonparametric test must be selected
parametric testparametric test......…assumes that the variable measured
is normally distributed in the population
…the data must represent an interval or ratio scale of measurement
…the selection of participants is independent
…the variances of the population comparison groups are equal
…a “more powerful” test in that it is more likely to reject a null hypothesis that is false, that is, the researcher is less likely to commit a Type II error
…used when the data represent a interval or ratio scale
nonparametric testnonparametric test......…makes no assumption about the
distribution of the variable in the population, that is, the shape of the distribution
…used when the data represent a nominal or ordinal scale, when a parametric assumption has been greatly violated, or when the nature of the distribution is not known
…a “less powerful” test in that it is less likely to reject a null hypothesis at a given level of significance
…usually requires a larger sample size to reach the same level of significance as a parametric test
The most common tests of significance…
t-testt-test
ANOVAANOVA
Chi SquareChi Square
t-testt-test......…used to determine whether two
means are significantly different at a selected probability level
…adjusts for the fact that the distribution of scores for small samples becomes increasingly different from the normal distribution as sample sizes become increasingly smaller
…the strategy of the t-test is to compare the actual mean difference observed to the difference expected by chance
…forms a ratio where the numerator is the difference between the sample means and the denominator is the chance difference that would be expected if the null hypothesis were true
…after the numerator is divided by the denominator, the resulting t value is compared to the appropriate t table value, depending on the probability level and the degrees of freedom
…if the t value is equal to or greater than the table value, then the null hypothesis is rejected because the difference is greater than would be expected due to chance
…there are two types of t-tests: the t-test for independent samples (randomly formed) and the t-test for nonindependent samples (nonrandomly formed, e.g., matching, performance on a pre-/post- test, different treatments)
ANOVAANOVA......…used to determine whether two or
more means are significantly different at a selected probability level
…avoids the need to compute duplicate t-tests to compare groups
…the strategy of ANOVA is that total variation, or variance, can be divided into two sources: a) treatment variance (“between groupsbetween groups,” variance caused by the treatment groups) and error variance (“within groupswithin groups” variance)
…forms a ratio, the FF ratio ratio, with the treatment variance as the numerator (between group variance) and error variance as the denominator (within group variance)
…the assumption is that randomly formed groups of participants are chosen and are essentially the same at the beginning of a study on a measure of the dependent variable
…at the study’s end, the question is whether the variance between the groups differs from the error variance by more than what would be expected by chance
…if the treatment variance is sufficiently larger than the error variance, a significant F ratio results, that is, the null hypothesis is rejected and it is concluded that the treatment had a significant effect on the dependent variable
…if the treatment variance is not sufficiently larger than the error variance, an insignificant F ratio results, that is, the null hypothesis is accepted and it is concluded that the treatment had no significant effect on the dependent variable
…when the F ratio is significant and more than two means are involved, researchers use multiple comparison procedures (e.g., Scheffé test, Tukey’s HSD test, Duncan’s multiple range test)
FANOVAFANOVA......…used when a research study uses
a factorial design to investigate two or more independent variables and the interactions between them
…provides a separate F ratio for each independent variable and each interaction
Multiple RegressionMultiple Regression......…a prediction equation that includes
more than one predictorpredictor
…predictors are variables known to individually predict (correlate with) the criterion to make a more accurate prediction
Chi Square (Chi Square (ΧΧ22))......…a nonparametric test of significance
appropriate for nominal or ordinal data that can be converted to frequencies
…compares the proportions actually observed (O) to the proportions expected (E) to see if they are significantly different
…the chi square value increases as the difference between observed and expected frequencies increases
…ANCOVA can also be used to increase the power of a statistical test by reducing within-group (error) variance, that is, to make a correct decision to reject the null hypothesis
One- and two- tailed tests of One- and two- tailed tests of significance...significance...
tests of significance that indicate the direction in which a difference may occur…the word “tail” indicates the area of
rejection beneath the normal curve
A = B……no difference between means; the
direction can be positive or negative
…direction can be in either tail of the normal curve
…called a “two-tailed” test
…divides the α level between the two tails of the normal curve
A > B or A < B……there is a difference between means;
the direction is either positive or negative
…called a “one-tailed” test
…the α level is found in one tail of the normal curve
Degrees of freedom (Degrees of freedom (dfdf)...)...
a statistical concept indicating that one degree of freedom is lost each time a population parameter is estimated on the basis of a sample of data from the population…indicates that there is no true
difference or relationship between parameters in the populations
the ability for the sample means to vary which is dependent upon the number of participants and the number of groups
for example: as the number of participants increases (df) the value needed to reject the null hypothesis becomes smaller
Mini-Quiz…Mini-Quiz…
True and false…
…inferential statistics are concerned with determining whether results obtained from a sample(s) are equivalent to those in the entire population
True
True and false…
…inferential statistics are used to make inferences about parameters, based on the statistics from a sample
True
True and false…
…inferential statistical analyses prove the results are either true or false
False
True and false…
…the word error in the term “standard error of the mean” indicates that the various sample means making up the distribution contain some error in their estimate of the population mean
True
True and false…
…purely by chance a researcher once in a while will select a sample that is quite different from the population
True
True and false…
…to find the mean of the sample means, the researcher adds up all of the sample means and divides by the number of means, as long as the size of each sample is the same
True
True and false…
…the size of the sample and the standard error of the mean negatively correlate
True
True and false…
…the difference between two sample means being a true or real difference means that the difference was caused by the dependent variable and not by chance
False
True and false…
…the null hypothesis states that any difference or relationship found for the samples is the result of sampling bias
False
True and false…
…the null hypothesis is the research hypothesis
False
True and false…
…tests of significance deal with probability not certainty
True
True and false…
…tests of significance enable the researcher to know for sure that the researcher’s analysis correct
False
True and false…
…the researcher makes the decision to reject or not reject the null hypothesis with a given probability of being correct
True
True and false…
…rejecting the null hypothesis represents the researcher’s conclusion that the means are significantly different
True
True and false…
…a significant difference between means indicates that they are too different to be the result of random, chance, sampling error
True
True and false…
…accepting the null hypothesis indicates that the means are determined not to be significantly different, that is, the difference is due to sampling error
True
True and false…
…researchers must always set the probability level, α, prior to testing for significance
False
True and false…
…testing for significance is actually a matter of comparing the consequences of making two possible incorrect decisions
True
True and false…
…with α = .05, the researcher believes the null hypothesis will be true 95% of the time
False
True and false…
…as a researcher decreases the chances of committing a Type I error, the researcher increases the probability of committing a Type II error
True
True and false…
…rejecting a null hypothesis at α = .001 proves the research hypothesis, that is, the independent variable causes the dependent variable
False
True and false…
…a “more powerful” statistical test of significance means that the researcher is less likely to commit a Type II error
True
True and false…
…a parametric test of significance should be used when the data represent an ordinal or nominal scale
False
True and false…
…generally speaking, a parametric test of significance should be used when the data represent interval or ratio scale
True
True and false…
…a significant F ratio indicates that there is at least one significant difference somewhere among the means but not which one it is
True
True and false…
…when many tests of statistical significance are performed, the probability level, α, tends to decrease because performing a large number of tests makes it more likely to obtain significant differences
False
True and false…
…when the chance of finding a significant difference between means is increased, so is the chance of committing a Type I error
True
Fill in the blank…
…an inferential statistic that tells the researcher how much the researcher would expect the sample means to differ if the researcher used other samples from the same population
standard error of the mean
Fill in the blank…
…a means by which researchers determine whether there is a significant of real difference between the sample means, one due not to random sampling error
tests of significance
Fill in the blank…
…the statement explaining that the difference between two sample means is the result of chance, random sampling error
null hypothesis
Fill in the blank…
…the type of error when the null hypothesis is true but the researcher concludes that it is false
Type I error
Fill in the blank…
…the type of error when the null hypothesis is false but the researcher concludes that it is true
Type II error
Fill in the blank…
…the term indicating the probability that the researcher is correct
level of significance
probability level
Fill in the blank…
…when α = .05, the probability that a difference is significant will be accurate within ___ standard deviations of the sample means (SEX)
+/- two SEX
Fill in the blank…
…when α = .01, the probability that a difference is significant will be accurate within ___ standard deviations of the sample means (SEX)
+/- three SEX
Fill in the blank…
…a null hypothesis which states that one difference can only occur in one direction requires a ____ test of significance
one-tailed
Fill in the blank…
…the type of error committed when a researcher does not reject a null hypothesis that should be rejected
Type II error
Fill in the blank…
…a statistical test of significance which determines whether the observed difference is sufficiently larger than a difference that would be expected solely by chance
t-test
Fill in the blank…
…multiple comparisons of the means that is decided upon before not after the study is conducted and is based upon research hypothesis
a priori comparisons
planned comparisons
Fill in the blank…
…the situation where multiple comparisons of the means cannot be decided upon before the study is conducted and is based upon research hypothesis
a posteriori comparison
post hoc comparison
Fill in the blank…
…the ability of a test of significance to reject a false null hypothesis, that is, to make a correct decision to reject the null hypothesis
power
This module has focused on...This module has focused on...
inferential statistics
...the statistical procedures for describing, synthesizing, analyzing, and interpreting quantitative data
The next module will focus on...The next module will focus on...
...the procedures for checking and storing all data in an organized manner and general guidelines for reporting findings
post-analysis considerationsand research reports