1Foundations of Research statistics module series
This should open as a PowerPoint “Show”.
If it does not please go to “slide show” and click “run show”
Please Click anywhere
2Foundations of Research Print
Want to print this module for note taking? When you open a PowerPoint module it saves a copy in your
downloads folder in "show" format (ppsx).
To convert it to a printing format (pptx): Click ‘esc’ to leave this open module
Open PowerPoint, browse to downloads, and open the module
Click ‘File” “Print’; In the dialogue box click “print what?”.
Select “Handouts (3 slides per page)”
Then, come back and re-run the show
Click here to skip ahead
Please Click anywhere
3Foundations of Research Welcome to the statistics module
series.
You are here
© Dr. David J. McKirnan, 2014The University of Illinois ChicagoDo not use or reproduce without [email protected]
Numbers & frequency distributions
Z and the normal distribution
Calculating a t score
Testing t: the Central Limit Theorem
Testing hypotheses: The critical ratio
4Foundations of Research
Module 2 introduced Z scores. Now we will see how to use them to evaluate data, and will introduce the crucial concept of critical ratio.
Using Z scores to evaluate data
Testing hypotheses: the critical ratio.
Evaluating data
5Foundations of Research Using Z to evaluate data
Z is at the core of how we use statistics to evaluate data.
Z indicates how far a score is from the M relative to the other scores in the sample.
Z combines… A score
The M of all scores in the sample
The variance in scores above and below M.
6Foundations of Research Using Z to evaluate data
Z is at the core of how we use statistics to evaluate data.
Z indicates how far a score is from the M relative to the other scores in the sample.
So… If X = 5.2
And M = 4
If S = 1.15
X - M = 1.2
7Foundations of Research Using Z to evaluate data
Z is at the core of how we use statistics to evaluate data.
Z indicates how far a score is from the M relative to the other scores in the sample.
So… If X = 5.2
And M = 4
If S = 1.15
X - M = 1.2
Z for our score is 1 (+).
5.2 – 41.15
= 1.05Z =X– M
S =
8Foundations of Research Using Z to evaluate data
Z is at the core of how we use statistics to evaluate data.
Z indicates how far a score is from the M relative to the other scores in the sample.
So… If X = 5.2
And M = 4
If S = 1.15
X - M = 1.2
This tells us that our score is higher than ~ 84% of the other scores in the distribution.
Z for our score is 1 (+).
9Foundations of Research Using Z to evaluate data
Z is at the core of how we use statistics to evaluate data.
Z indicates how far a score is from the M relative to the other scores in the sample.
This tells us that our score is higher than ~ 84% of the other scores in the distribution.
Unlike simple measurement with a ratio scale where a value – e.g. < 32o – has an absolute meaning.
…inferential statistics evaluates a score relative to a distribution of scores.
10Foundations of Research
-3 -2 -1 0 +1 +2 +3
Z Scores (standard deviation units)
34.13% of
cases
34.13% of
cases
13.59% of
cases
2.25% of
cases
13.59% of
cases
2.25% of
cases
50% of the scores in a distribution are above the M [Z = 0] 34.13% of the distribution
+13.59%
+2.25%...etc.
50% of scores are below the M
Z scores: areas under the normal curve, 2
0
11Foundations of Research
-3 -2 -1 0 +1 +2 +3Z Scores
(standard deviation units)
34.13% of
cases
34.13% of
cases
13.59% of
cases
2.25% of
cases
13.59% of
cases
2.25% of
cases
Z scores: areas under the normal curve, 2
84% of scores are below Z = 1
(One standard deviation above the Mean)
34.13% + 34.13%+ 13.59% + 2.25%...
+1
12Foundations of Research
-3 -2 -1 0 +1 +2 +3Z Scores
(standard deviation units)
34.13% of
cases
34.13% of
cases
13.59% of
cases
2.25% of
cases
13.59% of
cases
2.25% of
cases
Z scores: areas under the normal curve, 2
84% of scores are above Z = -1
(One standard deviation below the Mean)
-1
13Foundations of Research
-3 -2 -1 0 +1 +2 +3Z Scores
(standard deviation units)
34.13% of
cases
34.13% of
cases
13.59% of
cases
2.25% of
cases
13.59% of
cases
2.25% of
cases
Z scores: areas under the normal curve, 2
+2
98% of scores are less than Z = 2
Two standard deviations above the mean
13.59% + 34.13% + 34.13% + 13.59% + 2.25%…
14Foundations of Research
-3 -2 -1 0 +1 +2 +3Z Scores
(standard deviation units)
34.13% of
cases
34.13% of
cases
13.59% of
cases
2.25% of
cases
13.59% of
cases
2.25% of
cases
Z scores: areas under the normal curve, 2
-2
98% of scores are above Z = -2
15Foundations of Research
Evaluating Individual Scores
How good is a score of ‘6' in the group described in…
Table 1? Table 2?
Evaluate in terms of:
A. The distance of the score from the M.
B. The variance in the rest of the sample
C. Your criterion for a “significantly good” score
Scale Value0 1 2 3 4 5 6 7 8
0
1
2
3
4
5
Scale Value0 1 2 3 4 5 6 7 8
0
1
2
3
4
5
16Foundations of Research Using Z to compare scores
Table 1; high variance
Mean [M] = 4, Score (X) = 6
Standard Deviation (S) = 2.4
= 0.88Z =X - M
S = =
Table 1; low variance
Mean [M] = 4, Score (X) = 6
Standard Deviation (S) = 1.15
= 1.74
1. Calculate how far the score (X) is from the mean (M); X–M.
2. “Adjust” X–M by how much variance there is in the sample via standard deviation (S).
3. Calculate Z for each sample
6 - 42.4
22.4
Z = =X - M
S6 - 41.15
21.15 =
17Foundations of Research Using the normal distribution, 2
Table 1, high varianceX - M = 6 - 4 = 2
Standard Deviation (S) = 2.4.Z = (X – M / S) = (2 / 2.4) = 0.88
About 70% of participants are below this Z score
Table 2, low(er) varianceX - M = 6 - 4 = 2
Standard Deviation (S) = 1.15.Z = (X – M / S) = (2 / 1.15) = 1.74
About 90% of participants are below this Z score
B. The variance in the rest of the sample:Since Table 1 has more variance, a given score is not as good
relative to the rest of the scores.
A. The distance of the score from the M.
The participant is 2 units above the mean in both tables.
18Foundations of Research Comparing Scores: deviation x Variance
Scale Value0 1 2 3 4 5 6 7 8
0
1
2
3
4
5
Scale Value0 1 2 3 4 5 6 7 8
0
1
2
3
4
5
High variance(S = 2.4)
Less variance(S = 1.15)
‘6’ is not that high compared to rest of the distribution
Here ‘6’ is the highest score in the distribution
19Foundations of Research
Normal distribution; high variance
-3 -2 -1 0 +1 +2 +3Z Scores
(standard deviation units)
Z = .88
About 70% of cases
Table 1, high varianceX - M = 6 - 4 = 2
S = 2.4Z = (X – M / S) = (2 / 2.4) = 0.88
About 70% of participants are below this Z score
20Foundations of Research
-3 -2 -1 0 +1 +2 +3Z Scores
(standard deviation units)
Normal distribution; low variance
About 90% of cases
Z = 1.74
Table 2, low(er) varianceX - M = 6 - 4 = 2
S = 1.15.Z = (X – M / S) = (2 / 1.15) = 1.74
About 90% of participants are below this Z score
21Foundations of Research Evaluating scores using Z
-3 -2 -1 0 +1 +2 +3Z Scores
(standard deviation units)
X = 6, M = 4, S = 2.4, Z = .88
X = 6, M = 4, S = 1.15, Z = 1.74
70% of cases
90% of cases
C. Criterion for a “significantly good” score
If a “good” score is better than 90% of the sample…
..with high variance ’6' is
not so good,
with less variance ‘6’ is > 90% of the rest of the sample.
22Foundations of Research Summary: evaluating individual scores
A. The distance of the score from the M.
In both groups ‘6’ is two units > the M (X = 6, M = 4).
B. The variance in the rest of the sampleOne group has low variance and one has higher.
C. Criterion for “significantly good” scoreWhat % of the sample must the score be higher than…
How “good” is a score of ‘6' in two groups?
With low variance ‘6’ is higher relative to other scores then in a sample with higher variance.
23Foundations of Research Z / “standard” scores
Z scores (or standard deviation units) standardize scores by putting them on a common scale.
In our example the target score and M scores are the same, but come from samples with different variances.
We compare the target scores by translating them into Zs, which take into account variance.
Any scores can be translated into Z scores for comparison…
Using Z to standardize scores
24Foundations of Research
We cannot directly compare these scores because they are on different scales.
One is measured in hours & minutes, one in 10ths of a second.
We can use Z scores to change each scale to common metric i.e., as % of the larger distribution each score is above or below.
Z scores can be compared, since they are standardized by being relative to the larger population of scores.
Using Z to standardize scores, cont.
Which is “faster”; a 2:03:00 marathon,
or a 4 minute mile?
25Foundations of Research Comparing Zs
Location of 2:03 marathon on distribution; Z > 4
-3 -2 -1 0 +1 +2 +3Z Scores (standard deviation units)
4:30 4:25 4:20 4:10 4:00 3:50 3:45Mile times
Distribution of mile times, translated into Z scores
Location of 4 minute mile on distribution; Z = 1.
Distribution of world class marathon times as Z scores
-4 -3 -2 -1 0 +1 +2 +3 +4Z Scores (standard deviation units)
2:50 2:45 2:40 2:30 2:20 2:15 2:10Marathon times (raw scores)
A 2:03 marathon is “faster” than a 4 minute mile
26Foundations of Research Quiz 1
About what percentage of scores are below the line?
A. 45%
B. 66%
C. 84%
D. 16%
E. 50%
27Foundations of Research Quiz 1
About what percentage of scores are below the line?
A. 45%
B. 66%
C. 84%
D. 16%
E. 50%
Scores below the line are known as the “area under the curve”
The area under the curve below Z = 1 is 50% (below the M [0]) + ~34% (one standard deviation above the mean to the mean; Z = 1 Z = 0 ).
28Foundations of Research Quiz 1
About what is the likelihood of this score occurring by chance?
A. 45%
B. 66%
C. 84%
D. 16%
E. 50%
29Foundations of Research Quiz 1
About what is the likelihood of this score occurring by chance?
A. 45%
B. 66%
C. 84%
D. 16%
E. 50%
The “area under the curve” above z = 1 is ~14% (Z = 1 Z = 2) + ~2% (Z = 2 Z = 3).
The logic is that about 16% of scores will be higher than this score by chance alone.
30Foundations of Research Quiz 1
You got a score of 20 on your last exam.The M = 14, the maximum score = 25.Did you go well?
A. Of course; you are only 5 points from a perfect score.
B. No, your Tiger Mom will only accept 25/25.
C. Without the variance you cannot estimate how you did relative to your peers.
D. Midway between the average and the max. is at least a ‘C’, so I did OK.
31Foundations of Research Quiz 1
You got a score of 20 on your last exam.The M = 14, the maximum score = 25.Did you go well?
A. Of course; you are only 5 points from a perfect score.
B. No, your Tiger Mom will only accept 25/25.
C. Without the variance you cannot estimate how you did relative to your peers.
D. Midway between the average and the max. is at least a ‘C’, so I did OK.
If the exam is graded in absolute terms – if, say, the instructor sets “A’ at anything better than 80% - you are in.
32Foundations of Research Quiz 1
You got a score of 20 on your last exam.The M = 14, the maximum score = 25.Did you go well?
A. Of course; you are only 5 points from a perfect score.
B. No, your Tiger Mom will only accept 25/25.
C. Without the variance you cannot estimate how you did relative to your peers.
D. Midway between the average and the max. is at least a ‘C’, so I did OK.
Tough luck.
33Foundations of Research Quiz 1
You got a score of 20 on your last exam.The M = 14, the maximum score = 25.Did you go well?
A. Of course; you are only 5 points from a perfect score.
B. No, your Tiger Mom will only accept 25/25.
C. Without the variance you cannot estimate how you did relative to your peers.
D. Midway between the average and the max. is at least a ‘C’, so I did OK.
If your instructor is grading the way a statistician would, evaluating scores relative to the distribution (grading on the curve), you do not know.
You would need your score, the M score, and the Standard Deviation, i.e.: Z= 20 - 25 / S
34Foundations of Research Quiz 1
You got a score of 20 on your last exam.The M = 14, the maximum score = 25.Did you go well?
A. Of course; you are only 5 points from a perfect score.
B. No, your Tiger Mom will only accept 25/25.
C. Without the variance you cannot estimate how you did relative to your peers.
D. Midway between the average and the max. is at least a ‘C’, so I did OK.
Evaluating statistical outcomes always involves our setting a criterion for a “significantly good” score.
By convention we consider a research result as “significant” if it would have occurred less than 5% of the time by chance.
However, some have more lax criteria…
35Foundations of Research The critical ratio
Illustration of the nebular hypothesis (http://www.daviddarling.info/encyclopedia/N/nebhypoth.html)
Using Z scores to evaluate data
Testing hypotheses: the critical ratio.
36Foundations of ResearchUsing statistics to test
hypotheses:
Core concept:
No scientific finding is “absolutely” true.
Any effect is probabilistic:
We use empirical data to infer how the world words
We evaluate inferences by how likely the effect would
be to occur by chance.
We use the normal distribution to help us determine how likely an experimental outcome would be by chance alone.
37Foundations of Research Probabilities & Statistical Hypothesis Testing
Scientific observations are “innocent until proven guilty”.
If we compare two groups or test how far a score is from the mean, the odds of their being different by chance alone is always greater than 0.
We cannot just take any result and call it meaningful, since any result may be due to chance, not the Independent Variable.
So, we assume any result is by chance unless it is strong enough to be unlikely to occur randomly.
Null Hypothesis: All scores differ from the M by chance alone.
38Foundations of Research Probabilities & Statistical Hypothesis Testing
Using the Normal Distribution: More extreme scores have a lower probability of
occurring by chance alone Z = the % of cases above or below the observed score A high Z score may be “extreme” enough for us to reject
the null hypothesis
Null Hypothesis: All scores differ from the M by chance alone.
Alternate (experimental) hypothesis: This score differs from M by more than we would expect by chance…
39Foundations of Research “Statistical significance”
We assume a score with less than 5% probability of occurring
(i.e., higher or lower than 95% of the other scores… p < .05) is not by chance alone Z > +1.98 occurs < 95% of the time (p <.05).
If Z > 1.98 we consider the score to be “significantly” different from the mean
To test if an effect is “statistically significant”
Compute a Z score for the effect
Compare it to the critical value for p<.05; + 1.98
Statistical Significance
40Foundations of Research
-3 -2 -1 0 +1 +2 +3Z Scores
(standard deviation units)
34.13% of
cases
34.13% of
cases
13.59% of
cases
2.25% of
cases
13.59% of
cases
2.25% of
cases
2.4% of cases
2.4% of cases
Z = +1.98Z = -1.98
In a hypothetical distribution:
2.4% of cases are higher than Z = +1.98
2.4% of cases are lower than Z = -1.98
Statistical significance & Areas under the normal curve
95% of cases
With Z > +1.98 or < -1.98 we reject the null hypothesis & assume the results are not by chance alone.
Thus, Z > +1.98 or < -1.98 will occur < 5% of the time by chance alone.
41Foundations of Research Evaluating Research Questions
One participant’s score
The mean for a group
Means for 2 or more groups
Scores on two measured variables
Does this score differ from the M for the group by more than chance?
Does this M differ from the M for the general population by more than chance?
Is the difference between these Means more than we would expect by chance? -- more than the M difference between any 2 randomly selected groups?
Is the correlation (‘r’) between these variables more than we would expect by chance -- more than between any two randomly selected variables?
Data Statistical Question
42Foundations of Research Critical ratio
Critical ratio =
The strength of the results (experimental effect)
Amount of error variance (“noise” in the data)
To estimate the influence of chance we weight our results by the overall amount of variance in the data.
In “noisy” data (a lot of error variance) we need a very strong result to conclude that it was unlikely to have occurred by chance alone.
In very “clean” data (low variance) even a weak result may be statistically significant.
This is the Critical Ratio:
43Foundations of Research Critical ratio
Z is a basic Critical ratio
Distance of the score from the mean
Error variance or “noise” in the data
In our example the two samples had equally strong scores (X - M).
…but differed in the amount of variance in the distribution of scores
Weighting the effect – X - M – in each sample by it’s variance [S] yielded different Z scores: .88 v. 1.74.
This led us to different judgments of how likely each result would be to have occurred by chance.
Strength of the experimental result
Standard Deviation
Scale Value0 1 2 3 4 5 6 7 8
0
1
2
3
4
5
Scale Value0 1 2 3 4 5 6 7 8
0
1
2
3
4
5
44Foundations of Research
Applying the critical ratio to an experiment
Critical Ratio =Treatment Difference
Random Variance (Chance)
In an experiment the Treatment Difference is variance between the experimental and control groups.
Random variance or chance differences among participants within each group.
We evaluate that result by comparing it to a distribution of possible effects.
We estimate the distribution of possible effects based on the degrees of freedom (“df”).
We will get to these last 2 points in the next modules.
45Foundations of Research Examples of Critical Ratios
Z score =Individual Score – M for Group
Standard Deviation (S) for group
x Ms
t-test =Difference between group Ms
Standard Error of the Mean
F ratio =Between group differences (differences among > 3 group Ms)
Within Group differences (random variance among participants within groups)
r (correlation) =Random variance between participants within variables
Association between variables (joint Z scores) summed across participants (Zvariable1 x Zvariable2)
grp2
grp2
grp1
grp1
group2group1
n
Variance
n
Variance
MM
=
=
46Foundations of Research Quiz 2
Where would z or t have to fall for you to consider your results “statistically significant”? (Choose a color).
A.
B.
C.
D.
F.
47Foundations of Research Quiz 2
Where would z or t have to fall for you to consider your results “statistically significant”? (Choose a color).
A.
B.
C.
D.
F.
Both of these are correct.
A Z or t score greater than or less than 1.98 is consided it significant.
This means that the result would occur < 5% of the time by chance alone (p < 05).
48Foundations of Research Quiz 2
Where would z or t have to fall for you to consider your results “statistically significant”? (Choose a color).
A.
B.
C.
D.
F.
This value would also be statistically significant..
..it exceeds the .05% value we usually use, so it is a more conservative stnandard.
49Foundations of Research
Numbers are important for representing “reality” in science (and other fields).
Different measures of central tendency are useful & accurate for different data;
Mean is the most common.
Median useful for skewed data
Mode useful for simple categorical data
Variance (around the mean) is key to characterizing a set of numbers.
We understand a set of scores in terms of the:
Central tendency – the average or Mean score
The amount of variance in the scores, typically the Standard Deviation.
SummaryS
um
ma
ry
50Foundations of Research
Z is the prototype critical ratio:
Summary
Statistical decisions follow the critical ratio:
Distance of the score (X) from the mean (M)
Variance among all the scores in the sample [standard deviation (S)]
Z =X–M
S=
t is also a basic critical ratio used for comparing groups:
Difference between group Means
Variance within the two groups [standard error of the M (SE)]
t =M1 – M2=
grp2
grp2
grp1
grp1
n
Variance
n
Variance
Su
mm
ary
51Foundations of Research The critical ratio
The next module will show you how to derive a t value.
The last module in the series will describe the statistical logic of evaluating t scores