lecture 3 a brief review of some important statistical concepts
TRANSCRIPT
The Meaning of a Variable
A variable refers to any quantity that may take on more than one value Population is a variable because it is not fixed or constant –
changes over time The unemployment rate is a variable because it may take on any
value from 0-100%
A random variable can be thought of as an unknown value that may change every time it is inspected.
A random variable either may be discrete or continuous A variable is discrete if its possible values have jumps or breaks
Population - measured in integers or whole units: 1, 2, 3, … A variable is continuous if there are no jumps or breaks
Unemployment rate – needs not be measured in whole units: 1.77, .., 8.99, …
Descriptive Statistics
Descriptive statistics are used to describe the main features of a collection of data in quantitative terms.
Descriptive statistics aim to quantitatively summarize a data set
Some statistical summaries are especially common in descriptive analyses. For example
Frequency Distribution Central Tendency Dispersion Association
Frequency Distribution
Every set of data can be described in terms of how frequently certain values occur.
In statistics, a frequency distribution is a tabulation of the values that one or more variables take in a sample.
Consider the hypothetical prices of Dec CME Live Cattle Futures
Month Price (cents/lb)
May 67.05
June 66.89
July 67.45
August 68.39
September 67.45
October 70.10
November 68.39
Frequency Distribution
Univariate frequency distributions are often presented as lists ordered by quantity showing the number of times each value appears.
A frequency distribution may be grouped or ungrouped For a small number of observations - ungrouped frequency distribution For a large number of observations - grouped frequency distribution
Ungrouped
Price (X) Frequency
67.05 1
66.89 1
67.45 2
68.39 2
70.10 1
Grouped
Price (X) Frequency
65.00-66.99 1
67.00-68.99 4
69.00-70.99 1
71.00-72.99 0
73.00-74.99 0
Central Tendency
In statistics, the term central tendency relates to the way in which quantitative data tend to cluster around a “central value”.
A measure of central tendency is any of a number of ways of specifying this "central value.“
There are three important descriptive statistics that gives measures of the central tendency of a variable:
The Mean The Median The Mode
The Mean
The arithmetic mean is the most commonly-used type of average and is often referred to simply as the average.
In mathematics and statistics, the arithmetic mean (or simply the mean) of a list of numbers is the sum of all numbers in the list divided by the number of items in the list.
If the list is a statistical population, then the mean of that population is called a population mean.
If the list is a statistical sample, we call the resulting statistic a sample mean.
If we denote a set of data by X = (x1, x2, ..., xn), then the sample mean is typically denoted with a horizontal bar over the variable ( , enunciated "x bar").
The Greek letter μ is used to denote the arithmetic mean of an entire population.
X
The Sample Mean
In mathematical notation, the sample mean of a set of data denoted as X = (x1, x2, ..., xn) is given by
To calculate the mean, all of the observations (values) of X are added and the result is divided by the number of observations (n)
In the previous example, the mean price of Dec CME Live Cattle futures contract is
)...(11
211
n
n
ii XXXn
Xn
X
96.67)39.68...89.6605.67(7
11
1
n
iiXn
X
The Median
In statistics, a median is described as the numeric value separating the higher half of a sample or population from the lower half.
The median of a finite list of numbers can be found by arranging all the observations from lowest value to highest value and picking the middle one.
If there is an even number of observations, then there is no single middle value, so one often takes the mean of the two middle values.
Organize the price data in the previous example in ascending order
67.05, 66.89, 67.45, 67.45, 68.39, 68.39, 70.10
The median of this price series is 67.45
The Mode
In statistics, the mode is the value that occurs the most frequently in a data set.
The mode is not necessarily unique, since the same maximum frequency may be attained at different values.
Organize the price data in the previous example in ascending order
67.05, 66.89, 67.45, 67.45, 68.39, 68.39, 70.10
There are two modes in the given price data – 67.45 and 68.39
Thus the mode of the sample data is not unique
The sample price dataset may be said to be bimodal
A population or sample data may be unimodal, bimodal, or multimodal
Statistical Dispersion
In statistics, statistical dispersion (also called statistical variability or variation) is the variability or spread in a variable or probability distribution.
In particular, a measure of dispersion is a statistic (formula) that indicates how disperse (i.e., spread) the values of a given variable are
Common measures of statistical dispersion are The Variance, and The Standard Deviation
Dispersion is contrasted with location or central tendency, and together they are the most used properties of distributions
The Variance
In statistics, the variance of a random variable or distribution is the expected (mean) value of the square of the deviation of that variable from its expected value or mean.
Thus the variance is a measure of the amount of variation within the values of that variable, taking account of all possible values and their probabilities.
If a random variable X has the expected (mean) value E[X]=μ, then the variance of X can be given by:
22 ])[()( xXEXVar
The Variance
The above definition of variance encompasses random variables that are discrete or continuous. It can be expanded as follows:
22
22
222
22
22
2
])[(][
][
2][
][2][
]2[
])[()(
XEXE
XE
XE
XEXE
XXE
XEXVar
Variance is non-negative because the squares are positive or zero. The variance of a constant a is zero, and the variance of a variable
in a data set is 0 if and only if all entries have the same value.
Variance is invariant with respect to changes in a location parameter. That is, if a constant is added to all values of the variable, the variance is unchanged.
If all values are scaled by a constant, the variance is scaled by the square of that constant.
The Variance: Properties
0)( aVar
)()( XVaraXVar
)()(
)()(2
2
XVarabaXVar
XVaraaXVar
The Sample Variance
If we have a series of n measurements of a random variable X as Xi, where i = 1, 2, ..., n, then the sample variance, can be used to estimate the population variance of X = (x1, x2, ..., xn), The sample variance is calculated as
22
2
2
1
1
2
2
...1
11
XXXXXXn
n
XXS
n
n
ii
x
The Sample Variance
The denominator, (n-1) is known as the degrees of freedom in
calculating : Intuitively, once is known, only n-1 observation
values are free to vary, one is predetermined by
When n = 1 the variance of a single sample is obviously zero
regardless of the true variance. This bias needs to be corrected for
when n is small.
22
2
2
11
2
2 ...1
1
1XXXXXX
nn
XXS n
n
ii
x
X
2xs X
The Sample Variance
For the hypothetical price data for Dec CME Live Cattle futures contract, 67.05, 66.89, 67.45, 67.45, 68.39, 68.39, 70.10, the sample variance can be calculated as
24.1
96.6710.70...96.6705.6717
11
22
1
2
2
n
XXS
n
ii
x
The Standard Deviation
In statistics, the standard deviation of a random variable or distribution is the square root of its variance.
If a random variable X has the expected value (mean) E[X]=μ, then the standard deviation of X can be given by:
That is, the standard deviation σ (sigma) is the square root of the average value of (X − μ)2.
])[( 22 XExx
The Standard Deviation
If we have a series of n measurements of a random variable X as Xi, where i = 1, 2, ..., n, then the sample standard deviation, can be used to estimate the population standard deviation of X = (x1, x2, ..., xn). The sample standard deviation is calculated as
114.124.1
11
2
2
n
XXSS
n
ii
xx
The Mean Absolute Deviation
The mean or average deviation of X from its mean
is always zero. The positive and negative deviations cancel out in the summation, which makes it a useless measure of dispersion.
The mean absolute deviation (MAD), calculated by:
solves the “canceling out” problem.
n
XX
n
d ii )(
n
XX
n
d ii)(
The MSD and RMSD
The alternative way to address the canceling out problem is by squaring the deviations from the mean to obtain the mean squared deviation (MSD):
The problem of squaring can be solved by taking the square root of the MSD to obtain the root mean squared deviation (RMSD):
nXX
nd
ii
22
n
XXMSDRMSD
n
ii
1
2
RMSD vs. Standard Deviation
When calculating the RMSD, the squaring of the deviations gives a greater importance to the deviations that are larger in absolute value, which may or may not be desirable.
For statistical reasons, it turns out that a slight variation of the RMSD, known as the standard deviation (SX), is more desirable as a measure of dispersion.
n
XXMSDRMSD
n
ii
1
2
1
1
2
n
XXS
n
ii
x
Variance vs. MSDStandard Deviation vs. RMSD
Price (X) Mean (Xi−Mean) |Xi−Mean| |Xi−Mean|2
67.05 67.96 -0.91 0.91 0.8366.89 67.96 -1.07 1.07 1.1467.45 67.96 -0.51 0.51 0.2668.39 67.96 0.43 0.43 0.1867.45 67.96 -0.51 0.51 0.2670.10 67.96 2.14 2.14 4.5868.39 67.96 0.43 0.43 0.18Total 0.00 6.00 7.44
MAD = 0.86Variance = 1.24 MSD = 1.06Std. Dev. = 1.11 RMSD = 1.03
24
Association
Bivariate statistics can be used to examine the degree in which two variables are related or associated, without implying that one causes the other
Multivariate statistics can be used to examine the degree in which multiple variables are related or associated, without implying that one causes any or some of the others
Two common measures of bivariate and multivariate statistics are Covariance Correlation Coefficient
p 53
25
In Figure 3.3 (a) Y and X are positively but weakly correlated while in 3.3 (b) they are negatively and strongly correlated
p 54 Association: Bivariate Statistics
The covariance between two real-valued random variables X and Y, with mean (expected values) and , is
Cov(X, Y) can be negative, zero, or positive Random variables with covariance is zero are called uncorrelated
or independent
X vY
vYXE
vvvYXE
vXvEYEYXE
vvXYYXE
vYXEYYXXEYXCov
].[
].[
][][].[
].[
)]).([()]).([(),(
The Covariance
If X and Y are independent, then their covariance is zero. This follows because under independence,
Recalling the final form of the covariance derivation given above, and substituting, we get
The converse, however, is generally not true: Some pairs of random variables have covariance zero although they are not independent.
vYEXEYXE ][].[].[
Covariance
0),( vvYXCov
If X and Y are real-valued random variables and a and b are constants ("constant" in this context means non-random), then the following facts are a consequence of the definition of covariance:
),(),(
),(),(
),(),(
)(),(
0),(
YXCovbYaXCov
YXabCovbYaXCov
XYCovYXCov
XVarXXCov
aXCov
The Covariance: Properties
If X and Y are real-valued random variables and a and b are constants ("constant" in this context means non-random), then the following facts are a consequence of the definition of variance and covariance:
The variance of a finite sum of uncorrelated random variables is equal to the sum of their variances.
This is because, if X and Y are uncorrelated, their covariance is 0.
),(2)()()(
),(2)()()(22 YXabCovYVarbXVarabYaXVar
YXCovYVarXVarYXVar
Variance of the Sum of Correlated Random Variables
)()()( YVarXVarYXVar
30
The covariance is one measure of how closely the values taken by two variables X and Y vary together:
If we have a series of n measurements of X and Y written as Xi and Yi where i = 1, 2, ..., n, then the sample covariance can be used to estimate the population covariance between X=(X1, X2, …, Xn) and Y=(Y1, Y2, …, Yn). The sample covariance is calculated as
p 53
1n
YYXXn
1iii
,
yxS
The Sample Covariance
Correlation Coefficient
A disadvantage of the covariance statistic is that its magnitude can not be easily interpreted, since it depends on the units in which we measure X and Y
The related and more used correlation coefficient remedies this disadvantage by standardizing the deviations from the mean:
The correlation coefficient is symmetric, that is
YX
YXyx
YVarXVar
YXCov
.)()(
),( ,,
xyyx ,,
Correlation Coefficient
If we have a series of n measurements of X and Y written as Yi and Yi, where i = 1, 2, ..., n, then the sample correlation coefficient, can be used to estimate the population correlation coefficient between X and Y. The sample correlation coefficient is calculated as
yx
n
iii
yx SSn
YYXXr
)1(
))((1
,