text book : moore, mccabe and craig, introduction to the ... · pdf filetext book : moore,...
TRANSCRIPT
� Text Book : Moore, McCabe and Craig, Introduction to the Practice of Statistics, 6th ed.
� Class Web Page: http://www.stt.msu.edu/academics/classpages/
Choose STT421.
READ YOUR SYLLABUS WHICH IS ALSO POSTED ON THE CLASS PAGE!!!!!!!
Why are you taking this course?
a) It is required.
b) I am so bored that I enrolled the first course that I came across.
c) I heard that statistics can lie and I just wanted to check if it was true.
d) The name is funny and interesting.
e) Is it the statistics class that I am in? I will definitely drop this course as soon as this boring class ends.
f) I am aware of the fact that the science of statistics would really help me in my daily life let alone for my profession.
“It is easy to lie with statistics. It is hard to
tell the truth without it.”
Andrejs Dunkels
Looking at Data - DistributionsDisplaying Distributions with Graphs
IPS Chapter 1.1
© 2009 W.H. Freeman and Company
Objectives (IPS Chapter 1.1)
Displaying distributions with graphs
� Variables
� Types of variables
� Graphs for categorical variables
� Bar graphs
� Pie charts
� Graphs for quantitative variables
� Histograms
� Stemplots
� Stemplots versus histograms
� Interpreting histograms
� Time plots
What Is Statistics?
Why?1. Collecting Data
e.g., Survey
2. Presenting Data
e.g., Charts & Tables
3. Characterizing Data
e.g., Average
Data
Analysis
Decision-
Making
© 1984-1994 T/Maker Co.
© 1984-1994 T/Maker Co.
What Is Statistics?
Statistics is the science of data. It involves collecting, classifying, summarizing, organizing, analyzing, and interpreting numerical information.
Application Areas
�Economics
� Forecasting
� Demographics
�Sports
� Individual & Team
Performance
�Engineering
� Construction
� Materials
�Business
� Consumer Preferences
� Financial Trends
Statistical Methods
Statistical
Methods
Descriptive
Statistics
Inferential
Statistics
Descriptive Statistics
1. Involves
• Collecting Data
• Presenting Data
• Characterizing Data
2. Purpose
• Describe Data
X = 30.5 S2 = 113
0
25
50
Q1 Q2 Q3 Q4
$
1. Involves
• Estimation
• Hypothesis Testing
2. Purpose
• Make decisions about population characteristics
Inferential Statistics
Population?
What is a Statistical Inference?
� It is an estimate or prediction or some generalization about a population based on information contained in a sample.
Fundamental Elements
� Experimental unit(cases or individuals)
� Object upon which we collect data
� Example: people, animals, plants, or any object of interest.
� Population
� All items of interest
� Variable
� Characteristic of an individual experimental unit
� A variable varies among individuals.
� Example: age, height, blood pressure, ethnicity, leaf length, first language
� The distribution of a variable tells us what values the variable takes and how often it takes these values.
� Sample
� Subset of the units of a population
Two types of variables
� Variables can be either quantitative…
� Something that takes numerical values for which arithmetic operations,
such as adding and averaging, make sense.
� Example: How tall you are, your age, your blood cholesterol level, the
number of credit cards you own.
� … or categorical (qualitative).
� Something that falls into one of several categories. What can be counted
is the count or proportion of individuals in each category.
� Example: Your blood type (A, B, AB, O), your hair color, your ethnicity,
whether you paid income tax last tax year or not.
How do you know if a variable is categorical or quantitative?
Ask:
� What are the n individuals/units in the sample (of size “n”)?
� What is being recorded about those n individuals/units?
� Is that a number (� quantitative) or a statement (� categorical)?
Individualsin sample
DIAGNOSIS AGE AT DEATH
Patient A Heart disease 56
Patient B Stroke 70
Patient C Stroke 75
Patient D Lung cancer 60
Patient E Heart disease 80
Patient F Accident 73
Patient G Diabetes 69
QuantitativeEach individual is
attributed a numerical value.
CategoricalEach individual is assigned to one of several categories.
Ways to chart categorical data
� Summary Table
� Bar graphs
� Pie chart
� Lists categories & number of elements in category
� Obtained by tallying responses in category
� May show the class frequencies and/or relative
frequencies
Summary table
Example: Top 10 causes of death in the United States 2001
Rank Causes of death Counts% of top
10s% of total
deaths
1 Heart disease 700,142 37% 29%
2 Cancer 553,768 29% 23%
3 Cerebrovascular 163,538 9% 7%
4 Chronic respiratory 123,013 6% 5%
5 Accidents 101,537 5% 4%
6 Diabetes mellitus 71,372 4% 3%
7 Flu and pneumonia 62,034 3% 3%
8 Alzheimer’s disease 53,852 3% 2%
9 Kidney disorders 39,480 2% 2%
10 Septicemia 32,238 2% 1%
All other causes 629,967 21%
For each individual who died in the United States in 2001, we record what was
the cause of death. The table above is a summary of that information.
0
100
200
300
400
500
600
700
800
Hear
t dis
ease
s
Canc
ersC
erebr
ovas
cula
r
Chro
nic
resp
irato
ryA
ccid
ents
Dia
bete
s m
ellit
usFlu
& p
neum
onia
Alz
heim
er's d
isea
seK
idne
y dis
orde
rsS
eptic
emia
Counts
(x1000)
Top 10 causes of deaths in the United States 2001
Bar graphs
Each category is represented by one bar. The bar’s height shows the count (or
sometimes the percentage) for that particular category.
The number of individuals who died of an accident in
2001 is approximately 100,000.
0
100
200
300
400
500
600
700
800
Hear
t dis
ease
s
Canc
ersC
erebr
ovas
cula
r
Chro
nic
resp
irato
ryA
ccid
ents
Dia
bete
s m
ellit
usFlu
& p
neum
onia
Alz
heim
er's d
isea
seK
idne
y dis
orde
rsS
eptic
emia
Counts
(x1000)
Bar graph sorted by rank� Easy to analyze
Top 10 causes of deaths in the United States 2001
0
100
200
300
400
500
600
700
800
Acc
iden
ts
Alz
heim
er's d
isease
Canc
ersC
erebr
ovas
cula
r
Chroni
c re
spira
tory
Dia
bete
s m
ellit
usFlu
& p
neum
onia
Hear
t dis
ease
sK
idne
y diso
rder
sS
eptic
emia
Counts
(x1000) Sorted alphabetically
� Much less useful
Percent of people dying fromtop 10 causes of death in the United States in 2000
Pie chartsEach slice represents a piece of one whole. The size of a slice depends on what
percent of the whole this category represents. Angle size:(360°) (9%) = 32.4°
Make sure all percents add up to 100.
Bar Chart vs. Pie Chart
� Bar chart is used more often to represent the actual values while pie chart is used to represent relative proportions (in %).
� When comparison of relative proportion is important, pie chart is more appropriate.
� When the absolute counts or values are more important, a bar chart should be used.
Ways to chart quantitative data
� Histograms and stem-and-leaf plots (or stemplots)
These are summary graphs for a single variable. They are very useful to
understand the pattern of variability in the data.
� Line graphs: time plots
Use when there is a meaningful sequence, like time. The line connecting
the points helps emphasize any change over time.
Histograms
The range of values that a
variable can take is divided
into equal size intervals.
The histogram shows the
number of individual data
points that fall in each
interval.
The first column represents all states with a Hispanic percent in their
population between 0% and 4.99%. The height of the column shows how
many states (27) have a percent in this range.
The last column represents all states with a Hispanic percent in their
population between 40% and 44.99%. There is only one such state: New
Mexico, at 42.1% Hispanics.
Stem plots
How to make a stemplot:
1) Separate each observation into a stem, consisting of
all but the final (rightmost) digit, and a leaf, which is
that remaining final digit. Stems may have as many
digits as needed, but each leaf contains only a single
digit.
2) Write the stems in a vertical column with the smallest
value at the top, and draw a vertical line at the right
of this column.
3) Write each leaf in the row to the right of its stem, in
increasing order out from the stem.
STEM LEAVES
Example
Suppose we have the following data on weights (in lb) of 17 school-kids:
88 47 68 76 46 106 49 63 72 64 84 66
68 75 72 81 44
How do they work?
Sorted data:
44 46 47 49 63 64 66 68 68 72
72 75 76 81 84 88 106
Stem Leaf
4 4 6 7 9
5
6 3 4 6 8 8
7 2 2 5 6
8 1 4 8
9
10 6
key: 6|3 = 63
leaf unit: 1.0
stem unit: 10.0
S tate Percent
A labam a 1.5A laska 4.1A rizona 25.3A rkansas 2.8C alifo rn ia 32.4
C olorado 17.1C onnec ticut 9.4D elaw are 4.8Florida 16.8G eorg ia 5.3H aw aii 7.2
Idaho 7.9Illino is 10.7Ind iana 3.5Iow a 2.8K ansas 7
K entucky 1.5Louis iana 2.4M aine 0.7M aryland 4.3M assachusetts 6.8
M ichigan 3.3M innesota 2.9M iss iss ipp i 1.3M issouri 2.1M ontana 2N ebraska 5.5
N evada 19.7N ew H am psh ire 1.7N ew Jersey 13.3N ew M exico 42.1N ew York 15.1
N orthC aro lina 4.7N orthD akota 1.2O hio 1.9O klahom a 5.2O regon 8
P ennsylvan ia 3.2R hodeIs land 8.7S outhC aro lina 2.4S outhD akota 1.4Tennessee 2
Texas 32U tah 9V erm ont 0.9V irg in ia 4.7W ash ington 7.2W estV irg in ia 0.7
W iscons in 3.6W yom ing 6.4
Percent of Hispanic residents
in each of the 50 states
Step 2:
Assign the values to
stems and leaves
Step 1:
Sort the data
S ta te P e rc ent
M a in e 0 .7W es tV irg in ia 0 .7V erm on t 0 .9
N orth D ak o ta 1 .2M is s is s ip p i 1 .3
S ou th D ak o ta 1 .4A la bam a 1 .5K en tu c k y 1 .5
N ew H a m ps h ire 1 .7O h io 1 .9M o n ta na 2
Te nne s se e 2M is s ou ri 2 .1
Lo u is ia na 2 .4S ou th C aro lina 2 .4A rk an s as 2 .8
Iow a 2 .8M inn es o ta 2 .9P enn s ylvan ia 3 .2
M ic h iga n 3 .3Ind ian a 3 .5W is c o ns in 3 .6
A la s ka 4 .1M a ryland 4 .3
N orth C aro lina 4 .7V irg in ia 4 .7D e law are 4 .8
O k lah om a 5 .2G e org ia 5 .3N eb ra s k a 5 .5
W yom in g 6 .4M a s s ac hu se tts 6 .8K ans a s 7
H aw a ii 7 .2W as h ing ton 7 .2
Idah o 7 .9O reg on 8R ho de Is lan d 8 .7
U ta h 9C on nec tic u t 9 .4Ill ino is 10 .7
N ew J ers ey 13 .3N ew Yo rk 15 .1F lo rida 16 .8
C o lo rad o 17 .1N eva da 19 .7
A rizo na 25 .3Te xas 32C a lifo rn ia 32 .4
N ew M exic o 42 .1
Stem Plot
� To compare two related distributions, a back-to-back stem plot with common stems is useful.
� Stem plots do not work well for large datasets.
� When the observed values have too many digits, trim the numbers before making a stem plot.
� When plotting a moderate number of observations, you can spliteach stem.
� Unlike histograms, stem-and-leaf displays retain the original data.
� Stemplots are quick and dirty histograms that can easily be done by
hand.
� However, they are rarely found in scientific or laymen publications.
Stemplots versus histograms
Interpreting histograms
When describing the distribution of a quantitative variable, we look for the
overall pattern and for striking deviations from that pattern. We can describe
the overall pattern of a histogram by its shape, center, and spread.
Histogram with a line connecting
each column � too detailed
Histogram with a smoothed curve
highlighting the overall pattern of
the distribution
Most common distribution shapes
� A distribution is symmetric if the right and left
sides of the histogram are approximately mirror
images of each other.
Symmetric distribution
Skewed distribution
� A distribution is skewed to the right if the right
side of the histogram (side with larger values)
extends much farther out than the left side. It is
skewed to the left if the left side of the histogram
extends much farther out than the right side.
Alaska Florida
Outliers
An important kind of deviation is an outlier. Outliers are observations
that lie outside the overall pattern of a distribution. Always look for
outliers and try to explain them.
The overall pattern is fairly
symmetrical except for 2
states that clearly do not
belong to the main trend.
Alaska and Florida have
unusual representation of
the elderly in their
population.
A large gap in the
distribution is typically a
sign of an outlier.
A Few Questions
� How to choose the bin size? Let the computer decide it for
you. ☺
Line graphs: time plots
A trend is a rise or fall that
persists over time, despite
small irregularities.
In a time plot, time always goes on the horizontal, x axis.
We describe time series by looking for an overall pattern and for striking
deviations from that pattern. In a time series:
A pattern that repeats itself
at regular intervals of time is
called seasonal variation.
Retail price of fresh oranges
over time
This time plot shows a regular pattern of yearly variations. These are seasonal
variations in fresh orange pricing most likely due to similar seasonal variations in
the production of fresh oranges.
There is also an overall upward trend in pricing over time. It could simply be
reflecting inflation trends or a more fundamental change in this industry.
Time is on the horizontal, x axis.
The variable of interest—here
“retail price of fresh oranges”—
goes on the vertical, y axis.
1918 influenza epidemic
Date # Cases # Deaths
week 1 36 0week 2 531 0week 3 4233 130week 4 8682 552week 5 7164 738week 6 2229 414week 7 600 198week 8 164 90week 9 57 56week 10 722 50week 11 1517 71week 12 1828 137week 13 1539 178week 14 2416 194week 15 3148 290week 16 3465 310week 17 1440 149
01000
2000300040005000
600070008000
900010000
wee
k 1
wee
k 3
wee
k 5
wee
k 7
wee
k 9
wee
k 11
wee
k 13
wee
k 15
wee
k 17
# c
ase
s d
iag
no
se
d
0
100
200
300
400
500
600
700
800
# d
ea
ths r
ep
ort
ed
# Cases # Deaths
A time plot can be used to compare two or more
data sets covering the same time period.
The pattern over time for the number of flu diagnoses closely resembles that for the
number of deaths from the flu, indicating that about 8% to 10% of the people
diagnosed that year died shortly afterward, from complications of the flu.
Death rates f rom cancer (US, 1945-95)
0
50
100
150
200
250
1940 1950 1960 1970 1980 1990 2000
Years
Death
rate
(per
thousand)
Death rates from cancer (US, 1945-95)
0
50
100
150
200
250
1940 1960 1980 2000
Years
Death
rate
(per
thousand)
Death rates from cancer (US, 1945-95)
0
50
100
150
200
250
1940 1960 1980 2000
Years
Death
rate
(per
thousand)
A picture is worth a thousand words,
BUT
There is nothing like hard numbers.
���� Look at the scales.
Scales matter
How you stretch the axes and choose your scales can give a different impression.
Death rates from cancer (US, 1945-95)
120
140
160
180
200
220
1940 1960 1980 2000
Years
Death
rate
(per
thousand)
Looking at Data - DistributionsDescribing distributions with numbers
IPS Chapter 1.2
© 2009 W.H. Freeman and Company
Objectives (IPS Chapter 1.2)
Describing distributions with numbers
� Measures of center: mean, median, mode
� Mean versus median
� Measures of spread: quartiles, standard deviation
� Five-number summary and boxplot
� Choosing among summary statistics
� Changing the unit of measurement
Two Characteristics
The central tendency of the set of measurements–that is, the tendency of the data to cluster, or center, about certain numerical values.
Central Tendency
(Location)
Center
Two Characteristics
The variability (spread) of the set of measurements–that is, the spread of the data.
Variation
(Dispersion)
Sample A
Sample B
Variation of Sample B
Variation of Sample A
The mean or arithmetic average
To calculate the average, or mean, add
all values, then divide by the number of
individuals. It is the “center of mass.”
Sum of heights is 1598.3
divided by 25 women = 63.9 inches
58.2 64.0
59.5 64.5
60.7 64.1
60.9 64.8
61.9 65.2
61.9 65.7
62.2 66.2
62.2 66.7
62.4 67.1
62.9 67.8
63.9 68.9
63.1 69.6
63.9
Measure of center: the mean
x =1598.3
25= 63.9
Mathematical notation:
∑=
=n
iix
nx
1
1
woman(i)
height(x)
woman(i)
height(x)
i = 1 x1= 58.2 i = 14 x14= 64.0
i = 2 x2= 59.5 i = 15 x15= 64.5
i = 3 x3= 60.7 i = 16 x16= 64.1
i = 4 x4= 60.9 i = 17 x17= 64.8
i = 5 x5= 61.9 i = 18 x18= 65.2
i = 6 x6= 61.9 i = 19 x19= 65.7
i = 7 x7= 62.2 i = 20 x20= 66.2
i = 8 x8= 62.2 i = 21 x21= 66.7
i = 9 x9= 62.4 i = 22 x22= 67.1
i = 10 x10= 62.9 i = 23 x23= 67.8
i = 11 x11= 63.9 i = 24 x24= 68.9
i = 12 x12= 63.1 i = 25 x25= 69.6
i = 13 x13= 63.9 n= 25 ΣΣΣΣ=1598.3
Learn right away how to get the mean using your calculators.
n
xxxx n+++
=...21
Your numerical summary must be meaningful.
63.9x =
The distribution of women’s heights appears coherent and
symmetrical. The mean is a good numerical summary.
Measure of center: the medianThe median is the midpoint of a distribution—the number such
that half of the observations are smaller and half are larger.
1. Sort observations by size.n = number of observations
______________________________
1 1 0.6
2 2 1.2
3 3 1.6
4 4 1.9
5 5 1.5
6 6 2.1
7 7 2.3
8 8 2.3
9 9 2.5
10 10 2.8
11 11 2.9
12 3.3
13 3.4
14 1 3.6
15 2 3.7
16 3 3.8
17 4 3.9
18 5 4.1
19 6 4.2
20 7 4.5
21 8 4.7
22 9 4.9
23 10 5.3
24 11 5.6
n = 24 �n/2 = 12
Median = (3.3+3.4) /2 = 3.35
2.b. If n is even, the median is the mean of the two middle observations.
1 1 0.6
2 2 1.2
3 3 1.6
4 4 1.9
5 5 1.5
6 6 2.1
7 7 2.3
8 8 2.3
9 9 2.5
10 10 2.8
11 11 2.9
12 12 3.3
13 3.4
14 1 3.6
15 2 3.7
16 3 3.8
17 4 3.9
18 5 4.1
19 6 4.2
20 7 4.5
21 8 4.7
22 9 4.9
23 10 5.3
24 11 5.6
25 12 6.1
n = 25 (n+1)/2 = 26/2 = 13 Median = 3.4
2.a. If n is odd, the median is observation (n+1)/2 down the list
Mode
1. Measure of central tendency
2. Value that occurs most often
3. Not affected by extreme values
4. May be no mode or several modes
5. May be used for quantitative or qualitative data
Mode Example
� No ModeRaw Data: 10.3 4.9 8.9 11.7 6.3 7.7
� One ModeRaw Data: 6.3 4.9 8.9 6.3 4.9 4.9
� More Than 1 ModeRaw Data: 21 28 28 41 43 43
Modes
The peaks of a histogram are called modes.
A distribution is
� unimodal if it has one mode,
� bimodal if it has two modes,
� multimodal if it has three or more modes.
Unimodal, Bimodal or Multimodal?
Unimodal Bimodal Multimodal
Uniform Histogram
� A histogram that doesn’t appear to have any mode.
� All the bars are approximately the same.
Right-SkewedLeft-Skewed Symmetric
Mean = MedianMean Median Median Mean
Mode
Mode= Mode
The median, on the other hand,
is only slightly pulled to the right
by the outliers (from 3.4 to 3.6).
The mean is pulled to the
right a lot by the outliers
(from 3.4 to 4.2).
Pe
rce
nt o
f p
eo
ple
dyi
ng
Mean and median of a distribution with outliers
4.3=x
Without the outliers
2.4=x
With the outliers
Disease X:
Mean and median are the same.
Symmetric distribution…
4.3
4.3
=
=
M
x
Multiple myeloma:
5.2
4.3
=
=
M
x
… and a right-skewed distribution
The mean is pulled toward the skew.
Impact of skewed data
Spread of a Distribution
Are the values concentrated around the center of the distribution or they are spread out?
� Range,
� Interquartile Range,
� Variance,
� Standard Deviation.
Note: Variance and standard deviation are more appropriate when the distribution is symmetric.
Range
� Range of the data is defined as the difference between the maximum and the minimum values.
� Data: 23, 21, 67, 44, 51, 12, 35.Range = maximum – minimum = 67 – 12 = 55.
� Disadvantage: A single extreme value can make it very large, giving a value that does not really represent the data overall. On the other hand, it is not affected at all if some observation changes in the middle.
Interquartile Range (IQR)
� What is IQR?
IQR = Third Quartile (Q3) – First Quartile (Q1).
� What are quartiles?
Recall: Median divides the data into 2 equal halves.
The first quartile, median and the third quartile divide the data into 4 roughly equal parts.
M = median = 3.4
Q1= first quartile = 2.2
Q3= third quartile = 4.35
1 1 0.6
2 2 1.2
3 3 1.6
4 4 1.9
5 5 1.5
6 6 2.1
7 7 2.3
8 1 2.3
9 2 2.5
10 3 2.8
11 4 2.9
12 5 3.3
13 3.4
14 1 3.6
15 2 3.7
16 3 3.8
17 4 3.9
18 5 4.1
19 6 4.2
20 7 4.5
21 1 4.7
22 2 4.9
23 3 5.3
24 4 5.6
25 5 6.1
The quartiles
The first quartile, Q1, is the value in the
sample that has 25% of the data at or
below it (� it is the median of the lower
half of the sorted data, excluding M).
The third quartile, Q3, is the value in the
sample that has 75% of the data at or
below it (� it is the median of the upper
half of the sorted data, excluding M).
Outlier-sensitivity
� Data: 10, 13, 17, 21, 28, 32
Without the outlier� IQR = 15 Range = 22
� Data: 10, 13, 17, 21, 28, 32, 59
With the outlier� IQR = 19 Range = 49
Conclusion: IQR is less outlier-sensitive than range.
M = median = 3.4
Q3= third quartile = 4.35
Q1= first quartile = 2.2
25 6 6.1
24 5 5.6
23 4 5.3
22 3 4.9
21 2 4.7
20 1 4.5
19 6 4.2
18 5 4.1
17 4 3.9
16 3 3.8
15 2 3.7
14 1 3.6
13 3.4
12 6 3.3
11 5 2.9
10 4 2.8
9 3 2.5
8 2 2.3
7 1 2.3
6 6 2.1
5 5 1.5
4 4 1.9
3 3 1.6
2 2 1.2
1 1 0.6
Upper whisker=Largest = max. = 6.1
Lower whisker = Smallest = min = 0.6
Disease X
0
1
2
3
4
5
6
7
Ye
ars
un
til d
ea
th
Five-number summary:
min Q1 M Q3 max
Five-number summary and boxplot
BOXPLOT
� Data must be ordered from lowest value to highest value before finding the 5 number summary.
� Upper whisker of the box-plot extends to the largest value within the upper inner fence (Q3+1.5*IQR)
� Lower whisker of the box-plot extends to the smallest value within the lower inner fence (Q1-1.5*IQR)
0123456789
101112131415
Disease X Multiple Myeloma
Ye
ars
un
til d
ea
th
Comparing box plots for a normal and a right-skewed distribution
Boxplots for skewed data
Boxplots remain
true to the data and
depict clearly
symmetry or skew.
Suspected outliers
Outliers are troublesome data points, and it is important to be able to
identify them.
One way to raise the flag for a suspected outlier is to compare the
distance from the suspicious data point to the nearest quartile (Q1 or
Q3). We then compare this distance to the interquartile range
(distance between Q1 and Q3).
We call an observation a suspected outlier if it falls more than 1.5
times the size of the interquartile range (IQR) above the first quartile or
below the third quartile. This is called the “1.5 * IQR rule for outliers.”
Q3 = 4.35
Q1 = 2.2
25 6 7.9
24 5 6.1
23 4 5.3
22 3 4.9
21 2 4.7
20 1 4.5
19 6 4.2
18 5 4.1
17 4 3.9
16 3 3.8
15 2 3.7
14 1 3.6
13 3.4
12 6 3.3
11 5 2.9
10 4 2.8
9 3 2.5
8 2 2.3
7 1 2.3
6 6 2.1
5 5 1.5
4 4 1.9
3 3 1.6
2 2 1.2
1 1 0.6
Disease X
0
1
2
3
4
5
6
7
Ye
ars
un
til d
ea
th
8
Interquartile rangeQ3 – Q1
4.35 − 2.2 = 2.15
Distance to Q3
7.9 − 4.35 = 3.55
Individual #25 has a value of 7.9 years, which is 3.55 years above
the third quartile. This is more than 3.225 years, 1.5 * IQR. Thus,
individual #25 is a suspected outlier.
Histogram vs. Box plot
� Both histogram and box plot capture the symmetry or skewness of distributions.
� Box plot cannot indicate the modality of the data.
� Box plot is much better in finding outliers.
� The shape of histogram depends to some extent on the choice of bins.
Which type of car has the largest median Time to
accelerate?
A. upscale
B. sports
C. small
D. large
E. family
Which type of car always take less than 3.6
seconds to accelerate?
A. upscale
B. sports
C. small
D. Large
E. Luxury
Which type of car has the smallest IQR for
Time to accelerate?
A. upscale
B. sports
C. small
D. Large
E. Luxury
What is the shape of the distribution of
acceleration times for luxury cars?
A. Left skewed
B. Right skewed
C. Roughly symmetric
D. Cannot be determined from the information given.
What percent of luxury cars accelerate to 30 mph
in less than 3.5 seconds?
A. Roughly 25%
B. Exactly 37.5%
C. Roughly 50%
D. Roughly 75%
E. Cannot be determined from the information given
What percent of family cars accelerate to 30
mph in less than 3.5 seconds?
A. Less than 25%
B. More than 50%
C. Less than 50%
D. Exactly 75%
E. None of the above
The standard deviation “s” is used to describe the variation around the
mean. Like the mean, it is not resistant to skew or outliers.
2
1
2 )(1
1xx
ns
n
i −−
= ∑
1. First calculate the variance s2.
2
1
)(1
1xx
ns
n
i −−
= ∑
2. Then take the square root to get
the standard deviation s.
Measure of spread: the standard deviation
Mean± 1 s.d.
x
Calculations …
We’ll never calculate these by hand, so make sure to know how to get the standard deviation using your calculator or software.
2
1
)(1
xxdf
sn
i −= ∑
Mean = 63.4
Sum of squared deviations from mean = 85.2
Degrees freedom (df) = (n − 1) = 13
s2 = variance = 85.2/13 = 6.55 inches squared
s = standard deviation = √6.55 = 2.56 inches
Women height (inches)
i xi x (xi-x) (xi-x)2
1 59 63.4 -4.4 19.0
2 60 63.4 -3.4 11.3
3 61 63.4 -2.4 5.6
4 62 63.4 -1.4 1.8
5 62 63.4 -1.4 1.8
6 63 63.4 -0.4 0.1
7 63 63.4 -0.4 0.1
8 63 63.4 -0.4 0.1
9 64 63.4 0.6 0.4
10 64 63.4 0.6 0.4
11 65 63.4 1.6 2.7
12 66 63.4 2.6 7.0
13 67 63.4 3.6 13.3
14 68 63.4 4.6 21.6
Mean 63.4
Sum 0.0
Sum 85.2
Properties of Standard Deviation
� s measures spread about the mean and should be used only when the mean is the measure of center.
� s = 0 only when all observations have the same value and there is no spread. Otherwise, s > 0.
� s is not resistant to outliers.
� s has the same units of measurement as the original observations.
Choosing among summary statistics
� Because the mean is not
resistant to outliers or skew, use
it to describe distributions that are
fairly symmetrical and don’t have
outliers.
� Plot the mean and use the
standard deviation for error bars.
� Otherwise use the median in the
five number summary which can
be plotted as a boxplot.
Height of 30 Women
58
59
60
61
62
63
64
65
66
67
68
69
Box Plot Mean +/- SD
He
igh
t in
In
ch
es
Boxplot Mean ± SD
Changing the unit of measurement
Variables can be recorded in different units of measurement. Most
often, one measurement unit is a linear transformation of another
measurement unit: xnew = a + bx.
Temperatures can be expressed in degrees Fahrenheit or degrees Celsius.
TemperatureFahrenheit = 32 + (9/5)* TemperatureCelsius � a + bx.
Linear transformations do not change the basic shape of a distribution
(skew, symmetry, multimodal). But they do change the measures of
center and spread:
� Multiplying each observation by a positive number b multiplies both
measures of center (mean, median) and spread (IQR, s) by b.
� Adding the same number a (positive or negative) to each observation
adds a to measures of center and to quartiles but it does not change
measures of spread (IQR, s).
� Temperature data (in F): 10, 13, 18, 22, 29� Range = 19 F, IQR =14 F, s = 7.5 F,
s2 = 56.25 F2.� Suppose transformed data = (-3)*original data.� So transformed data (in F): -30, -39, -54, -66, -87� Range = |-3|*19 = 57 F, IQR = |-3|*14 = 42 F, s = |-3|* 7.5 = 22.50 F, s2 = (-3)2*56.25 = 506.25 F2.
� Temperature data (in F): 10, 13, 18, 22, 29
� Range = 19 F, IQR =14 F, s = 7.5 F, s2 = 56.25 F2.
� Suppose transformed data = original data + 2.5.
� Hence transformed data (in F): 12.5, 15.5, 20.5, 24.5, 31.5
� Range = 19 F, IQR =14 F, s = 7.5 F, s2 = 56.25 F2.
Software output for summary statistics:
Excel - From Menu:
Tools → Data Analysis
→Descriptive Statistics
Give common
statistics of your
sample data.
Minitab
MINITAB
For graphs
From the Menu
�Graphs→Histogram
�Graphs → Stem-and-Leaf
�Graphs → Box-Plot
For measures of central tendency and spread
� Stat → Basic Statistics → Display Descriptive
Statistics
Comparing Distributions
Use of Stem-and-Leaf Display
and
5 number summary
Comparing Distributions: Use of Stem-and-Leaf Plot
East Lansing Gas Prices
2.53
2.59
2.69
2.61
2.49
2.49
2.56
2.74
2.74
2.76
2.69
2.53
2.49
2.68
Ann Arbor Gas Prices
2.69
2.69
2.72
2.94
2.69
2.95
2.69
2.72
2.63
2.89
2.75
2.96
2.93
2.69
2.89
Ann Arbor
Gas Prices
East Lansing
Gas Prices
24|9 = $2.49
Gas prices in Ann
Arbor are unimodal
with right skew.
They are between
$2.63 and $2.96 per
gallon, centered at
about $2.75 per
gallon
Gas prices in
East Lansing
are roughly
uniform
between $2.49
and $2.76 per
gallon,
centered at
$2.60 per
gallon
Gas prices tend to be
higher in Ann Arbor and
have more variability. The
range of prices in Ann
Arbor is 33¢ and in East
Lansing is only 27¢
Ann Arbor
Gas Prices
East Lansing
Gas Prices
24|9 = $2.49
Min Q1 Median Q3 Max
Ann Arbor Gas Prices $2.63 $2.69 $2.72 $2.93 $2.96
East Lansing Gas Prices $2.49 $2.53 $2.60 $2.69 $2.76
5-Number Summary
Comparing Distributions
Use of Histograms
Which data have more variability?
A. Graph A
B. Graph B
C. Both have the same variability
0 2 4 6 8 10
1
2
3
4
5
12
6
SCORE
1
2
3
4
5
6
FR
EQ
UE
NC
Y
0 2 4 6 8 10 12SCORE
AB
Which data have more variability?
A. Graph A
B. Graph B
C. Both have the same variability
AB
0 2 4 6 8 10
1
2
3
4
5
12
6
SCORE
1
2
3
4
5
6
0 2 4 6 8 10 12SCORE
Which data have a higher median?
A. Graph A
B. Graph B
C. Both have the same median
AB
0 2 4 6 8 10
1
2
3
4
5
12
6
SCORE
1
2
3
4
5
6
0 2 4 6 8 10 12SCORE
88
Which data have more variability?
A. Graph A
B. Graph B
C. Roughly, both have the same variability
AB
0 2 4 6 8 10
1
2
3
4
5
12
6
SCORE
1
2
3
4
5
6
FR
EQ
UE
NC
Y
0 2 4 6 8 10 12SCORE
89
Looking at Data - DistributionsDensity Curves and Normal Distributions
IPS Chapter 1.3
© 2009 W.H. Freeman and Company
Objectives (IPS Chapter 1.3)
Density curves and Normal distributions
� Density curves
� Measuring center and spread for density curves
� Normal distributions
� The 68-95-99.7 rule
� Standardizing observations
� Using the standard Normal Table
� Inverse Normal calculations
� Normal quantile plots
Density curvesA density curve is a mathematical model of a distribution.
The total area under the curve, by definition, is equal to 1, or 100%.
The area under the curve for a range of values is the proportion of all observations for that range.
Histogram of a sample with the
smoothed, density curve
describing theoretically the
population.
Density curves come in any
imaginable shape.
Some are well known
mathematically and others aren’t.
Median and mean of a density curve
The median of a density curve is the equal-areas point: the point that
divides the area under the curve in half.
The mean of a density curve is the balance point, at which the curve
would balance if it were made of solid material.
The median and mean are the same for a symmetric density curve.
The mean of a skewed curve is pulled in the direction of the long tail.
Normal distributions
e = 2.71828… The base of the natural logarithm
π = pi = 3.14159…
Normal – or Gaussian – distributions are a family of symmetrical, bell-
shaped density curves defined by a mean µ (mu) and a standard
deviation σ (sigma) : N(µ,σ).
21
21( )
(2 )
x
f x e
µσ
σ π
− − =
xxxxxxxx
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30
A family of density curves
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30
Here, means are different
(µ = 10, 15, and 20) while standard
deviations are the same (σ = 3)
Here, means are the same (µ = 15)
while standard deviations are
different (σ = 2, 4, and 6).
mean µ = 64.5 standard deviation σσσσ = 2.5
N(µ, σσσσ) = N(64.5, 2.5)
Reminder: µ (mu) is the mean of the idealized curve, while is the mean of a sample.
σ (sigma) is the standard deviation of the idealized curve, while s is the s.d. of a sample.
The 68-95-99.7% Rule for Normal Distributions
� About 68% of all observations
are within 1 standard deviation
(σ) of the mean (µ).
� About 95% of all observations
are within 2 σ of the mean µ.
� Almost all (99.7%) observations
are within 3 σ of the mean.
Inflection point
x
Because all Normal distributions share the same properties, we can
standardize our data to transform any Normal curve N(µ,σ) into the
standard Normal curve N(0,1).
The standard Normal distribution
For each x we calculate a new value, z (called a z-score).
N(0,1)
=>
zx
N(64.5, 2.5)
Standardized height (no units)
How to compare apples with oranges?
� A college admissions committee is looking at the files of two candidates, one with a total SAT score of 1500 and another with an ACT score of 22. Which candidate scored better?
� How do we compare things when they are measured on different scales?
� We need to standardize the values so that they will be free of units.
z =(x − µ )
σ
A z-score measures the number of standard deviations that a data
value x is from the mean µ.
Standardizing: calculating z-scores
� When x is larger than the mean, z is positive.
� When x is smaller than the mean, z is negative.
1 , ==−+
=+=σσ
σµσµ
σµ zxfor
When x is 1 standard deviation larger
than the mean, then z = 1.
222
,2 ==−+
=+=σσ
σµσµ
σµ zxfor
When x is 2 standard deviations larger
than the mean, then z = 2.
Interpretation of z-scores
� The z-scores measure the distance of the data values from the mean in the standard deviation scale.
� A z-score of 1 means that data value is 1 standard deviation above the mean.
� A z-score of -1.2 means that data value is 1.2 standard deviations below the mean.
� Regardless of the direction, the further a data value is from the mean, the more unusual it is.
� A z-score of -1.3 is more unusual than a z-score of 1.2.
z-scores: An Example
Data: 4, 3, 10, 12, 8, 9, 3 (n=7 in this case)Mean = (4+3+10+12+8+9+3)/7 = 49/7 =7.Standard Deviation = 3.65.
Original Value z-score --------------------------------------------------------------
4 (4 – 7)/3.65 = -0.823 (3 – 7)/3.65 = -1.10
10 (10 – 7)/3.65 = 0.82 12 (12 – 7)/3.65 = 1.37 8 (8 – 7)/3.65 = 0.279 (9 – 7)/3.65 = 0.553 (3 – 7)/3.65 = -1.10
--------------------------------------------------------------
How to use z-scores?
� A college admissions committee is looking at the files of two candidates, one with a total SAT score of 1500 and another with an ACT score of 22. Which candidate scored better?
� SAT score mean = 1600, std dev = 500.
� ACT score mean = 23, std dev = 6.
� SAT score 1500 has z-score = (1500-1600)/500 = -0.2.
� ACT score 22 has z-score = (22-23)/6 = -0.17.
� ACT score 22 is better than SAT score 1500.
Which is more unusual?A. A 58 in tall woman
z-score = (58-63.6)/2.5 = -2.24.B. A 64 in tall man
z-score = (64-69)/2.8 = -1.79.C. They are the same.
Heights of adult men have
mean of 69.0 in.
std. dev. of 2.8 in.
Heights of adult women have
mean of 63.6 in.
std. dev. of 2.5 in.
Using z-scores to solve problems
An example using height data and U.S. Marine and Army height requirements
Question: Are the height restrictions set up by the
U.S. Army and U.S. Marine more restrictive for
men or women or are they roughly the same?
Heights of adult women have � mean of 63.6 in.
� standard deviation of 2.5 in.
Data from a National Health Survey
Heights of adult men have– mean of 69.0 in. – standard deviation of 2.8 in.
Men Minimum
Women Minimum
U.S. Army 60 in 58 in
U.S. Marine Corps 64 in 58 in
Height Restrictions
Heights of adult men have � mean of 69.0 in.
� standard deviation of 2.8 in.
Heights of adult women have – mean of 63.6 in.– standard deviation of 2.5 in.
Men Minimum Women minimum
U.S.
Army
U.S.
Marine
60 in
z-score = -3.21
Less restrictive
58 in
z-score = -2.24
More restrictive
64 in
z-score = -1.79
More restrictive
58 in
z-score = -2.24
Less restrictive
Effect of Standardization
� Standardization into z-scores does not change the shape of the histogram.
� Standardization into z-scores changes the center of the distribution by making the mean 0.
� Standardization into z-scores changes the spread of the distribution by making the standard deviation 1.
mean µ = 64.5"
standard deviation σ = 2.5"
x (height) = 67"
We calculate z, the standardized value of x:
mean from dev. stand. 1 15.2
5.2
5.2
)5.6467( ,
)(=>==
−=
−= z
xz
σµ
Because of the 68-95-99.7 rule, we can conclude that the percent of women
shorter than 67” should be, approximately, .68 + half of (1 - .68) = .84 or 84%.
Area= ???
Area = ???
N(µ, σ) = N(64.5, 2.5)
µµµµ = 64.5” x = 67”
z = 0 z = 1
Ex. Women heights
Women’s heights follow the N(64.5”,2.5”)
distribution. What percent of women are
shorter than 67 inches tall (that’s 5’6”)?
Using the standard Normal table
(…)
Table A gives the area under the standard Normal curve to the left of any z value.
.0082 is the
area under
N(0,1) left
of z = -
2.40
.0080 is the area
under N(0,1) left
of z = -2.41
0.0069 is the area
under N(0,1) left
of z = -2.46
Area ≈ 0.84
Area ≈ 0.16
N(µ, σ) =
N(64.5”, 2.5”)
µ = 64.5” x = 67” z = 1
Conclusion:
84.13% of women are shorter than 67”.
By subtraction, 1 - 0.8413, or 15.87% of
women are taller than 67".
For z = 1.00, the area under
the standard Normal curve
to the left of z is 0.8413.
Percent of women shorter than 67”
Tips on using Table A
Because the Normal distribution is
symmetrical, there are 2 ways
that you can calculate the area
under the standard Normal curve
to the right of a z value.
area right of z = 1 - area left of z
Area = 0.9901
Area = 0.0099
z = -2.33
area right of z = area left of -z
Tips on using Table A
To calculate the area between 2 z- values, first get the area under N(0,1)
to the left for each z-value from Table A.
area between z1 and z2 =
area left of z1 – area left of z2
A common mistake made by
students is to subtract both z
values. But the Normal curve
is not uniform.
Then subtract the
smaller area from the
larger area.
���� The area under N(0,1) for a single value of z is zero.
(Try calculating the area to the left of z minus that same area!)
Calculating the area for normal
distribution by TI calculators
Use TI 83/84 Plus.
� Press [2nd] & [VARS] (i.e. [DISTR])
� Select 2: normalcdf
� Format of command:
normalcdf(lower bound, upper bound, mean, std.dev.)
� For percentage between a and b
normalcdf(a, b, mean, sigma)
� For area (percentage) below a specific value b
normalcdf(-1000, b, mean, sigma)
� For area (percentage) above any specific value a
normalcdf( a, 1000, mean, sigma)
Approximately what percent of U.S. women do you expect to be
less than 64 in tall?
� Heights of adult women are normally distributed with
� mean of 63.6 in,
� standard deviation of 2.5 in.
� Note that here upper bound is 64, but there is no mention of lower bound.
� So take a very small value for lower bound, say -1000.
For this problem
normalcdf(-1000, 64, 63.6, 2.5) = 0.5636.
i.e. about 56.4% of adult U.S. women have heights less than 64 in.
The National Collegiate Athletic Association (NCAA) requires Division I athletes to
score at least 820 on the combined math and verbal SAT exam to compete in their
first college year. The SAT scores of 2003 were approximately normal with mean
1026 and standard deviation 209.
What proportion of all students would be NCAA qualifiers (SAT ≥ 820)?
820
1026
209
( )
(820 1026)
209
2060.99
209
Table A: area under
N(0,1) to the left of
z = -0.99 is 0.1611
or approx. 16%.
x
xz
z
z
µ
σ
µσ
=
=
=
−=
−=
−= ≈ −
area right of 820 = total area - area left of 820= 1 - 0.1611
≈ 84%
normalcdf (820,1000,1026,209)
The NCAA defines a “partial qualifier” eligible to practice and receive an athletic
scholarship, but not to compete, with a combined SAT score of at least 720.
What proportion of all students who take the SAT would be partial qualifiers?
That is, what proportion have scores between 720 and 820?
720
1026
209
( )
(720 1026)
209
3061.46
209
Table A: area under
N(0,1) to the left of
z = -.99 is -1.46
or approx. 7%.
x
xz
z
z
µ
σ
µσ
=
=
=
−=
−=
−= ≈ −
About 9% of all students who take the SAT have scores
between 720 and 820.
area between = area left of 820 - area left of 720
720 and 820 = 0.1611 - 0.0721
≈ 9%
normalcdf (720,820,1026,209)
N(0,1)
z =(x − µ )
σ
The cool thing about working with normally distributed data is that we can manipulate it, and then find answers to questions that involve comparing seemingly non-comparable distributions.
We do this by “standardizing” the data. All this involves is changing the scale so that the mean now = 0 and the standard deviation =1. If you do this to different distributions it makes them comparable.
180 200 220 240 260 280 300 320
Gestation time (days)
Vitamins only Vitamins and better food
What is the effect of better maternal care on gestation time and preemies?
The goal is to obtain pregnancies 240 days (8 months) or longer.
µ 250σ 20
µ 266σ 15
Ex. Gestation time in malnourished mothers
What improvement did we get
by adding better food?
240
250
20
( )
(240 250)
20
100.5
20
(half a standard deviation)
Table A: area under N(0,1) to
the left of z = -0.5 is 0.3085.
x
xz
z
z
µ
σ
µσ
=
=
=
−=
−=
−= = −
Vitamins Only
Under each treatment, what percent of mothers failed to carry their babies at
least 240 days?
190 210 230 250 270 290 310
Gestation time (days)
Vitamins only: 30.85% of women
would be expected to have gestation
times shorter than 240 days.
µ=250, σ=20, x=240
normalcdf(-1000,240,250,20)
240
266
15
( )
(240 266)
15
261.73
15
(almost 2 sd from mean)
Table A: area under N(0,1) to
the left of z = -1.73 is 0.0418.
x
xz
z
z
µ
σ
µσ
=
=
=
−=
−=
−= = −
Vitamins and better food
221 236 251 266 281 296 311
Gestation time (days)
Vitamins and better food: 4.18% of women
would be expected to have gestation
times shorter than 240 days.
µ=266, σ=15, x=240
Compared to vitamin supplements alone, vitamins and better food resulted in a much
smaller percentage of women with pregnancy terms below 8 months (4% vs. 31%).
normalcdf(-1000,240,266,15)
Inverse normal calculations
We may also want to find the observed range of values that correspond
to a given proportion/ area under the curve.
For that, we use Table A backward:
� we first find the desired
area/ proportion in the
body of the table,
� we then read the
corresponding z-value
from the left column and
top row.
For an area to the left of 1.25 % (0.0125), the z-value is -2.24
Use TI 83/84 Plus.
� Press [2nd] & [VARS] (i.e. [DISTR])
� Select 3: invNorm
� Format of command:
invNorm(fraction, mean, std.dev.)
� invNorm only considers percentage or fraction in the lower tail of normal distribution.
25695.255
)15*67.0(266
)*()(
0.67.-about is N(0,1)
under 25% arealower
for the valuez :A Table
?
%25arealower
%75areaupper
15
266
≈=
−+=
+=⇔−
=
=
=
=
=
=
x
x
zxx
z
x
σµσ
µ
σ
µ
Vitamins and better food
221 236 251 266 281 296 311
Gestation time (days)
µ=266, σ=15, upper area 75%
How long are the longest 75% of pregnancies when mothers with malnutrition are
given vitamins and better food?
?
upper 75%
� The 75% longest pregnancies in this group are about 256 days or longer.
Remember that Table A gives the area to the
left of z. Thus, we need to search for the lower
25% in Table A in order to get z. invnorm(0.25, 266,15)=256
One way to assess if a distribution is indeed approximately normal is to
plot the data on a normal quantile plot.
The data points are ranked and the percentile ranks are converted to z-
scores with Table A. The z-scores are then used for the x axis against
which the data are plotted on the y axis of the normal quantile plot.
� If the distribution is indeed normal the plot will show a straight line,
indicating a good match between the data and a normal distribution.
� Systematic deviations from a straight line indicate a non-normal
distribution. Outliers appear as points that are far away from the overall
pattern of the plot.
Normal quantile plots
Normal quantile plots are complex to do by hand, but they are standard
features in most statistical software.
Good fit to a straight line: the
distribution of rainwater pH
values is close to normal.
Curved pattern: the data are not
normally distributed. Instead, it shows
a right skew: a few individuals have
particularly long survival times.
By Minitab
� Graph-Probability plot-Single (For one sample)