statistics sampling distributions and point estimation of parameters contents, figures, and...
Post on 18-Jan-2018
227 Views
Preview:
DESCRIPTION
TRANSCRIPT
StatisticsStatisticsSampling Distributions and Point Estimation of Parameters
Contents, figures, and exercises come from the textbook: Applied Statistics and Probability for Engineers, 5th Edition, by Douglas C. Montgomery, John Wiley & Sons, Inc., 2011.
StatisticA function of observations, ,
,…,Also a random variableSample meanSample variance Its probability distribution is
called a sampling distribution
Point EstimationPoint Estimation
Contents, figures, and exercises come from the textbook: Applied Statistics and Probability for Engineers, 5th Edition, by Douglas C. Montgomery, John Wiley & Sons, Inc., 2011.
1X 2X nX
X
2S
Point estimator of◦A statistic
Point estimate of◦A particular numerical value
Contents, figures, and exercises come from the textbook: Applied Statistics and Probability for Engineers, 5th Edition, by Douglas C. Montgomery, John Wiley & Sons, Inc., 2011.
),...,,(ˆ21 nXXXh
Mean ◦The estimate
Variance ◦The estimate
Proportion◦The estimate◦ is the number of items that belong to the class
of interestDifference in means,
◦The estimateDifference in two proportions
◦The estimate
x2
22ˆ sp
nxp /ˆ
x
21
2121 ˆˆˆˆ xx
21 pp
21 ˆˆ pp
Sampling Distributions and Sampling Distributions and the Central Limit Theoremthe Central Limit TheoremRandom sample
◦The random variables , ,…, are a random sample of size if (a) the ‘s are independent random variables, and (b) every has the same probability distribution
1X nX2Xn iX
iX
If , ,…, are normally and independently distributed with mean and variance◦ has a normal
distribution ◦with mean
◦variance
Contents, figures, and exercises come from the textbook: Applied Statistics and Probability for Engineers, 5th Edition, by Douglas C. Montgomery, John Wiley & Sons, Inc., 2011.
1X nX2X
nXXXX n
21
2
nX
nnX
2
2
2222
Central Limit Theorem◦If , ,…, is a random sample of
size taken from a population (either finite or infinite) with mean and finite variance , and if is the sample mean, the limiting form of the distribution of
◦as , is the standard normal distribution.
Works when◦ , regardless of the shape of the
population◦ , if not severely nonnormal
1X nX2X n
2X
nXZ
/
n
30n
30n
Two independent populations with means and , and variances and
◦is approximately standard normal, or◦is exactly standard normal if the two
populations are normal
Contents, figures, and exercises come from the textbook: Applied Statistics and Probability for Engineers, 5th Edition, by Douglas C. Montgomery, John Wiley & Sons, Inc., 2011.
21
21
22
2221
21
2121
//
)(
nn
XXZ
Example 7-1 Resistors◦An electronics company manufactures
resistors that have a mean resistance of 100 ohms and a standard deviation of 10 ohms. The distribution of resistance is normal. Find the probability that a random sample of resistors will have an average resistance less than 95 ohms
Example 7-2 Central Limit Theorem◦ Suppose that has a continuous uniform
distribution
◦Find the distribution of the sample mean of a random sample of size
25n
X
otherwise0
642/1)(
xxf
40n
Example 7-3 Aircraft Engine Life◦ The effective life of a component used in a jet-
turbine aircraft engine is a random variable with mean 5000 hours and standard deviation 40 hours. The distribution of effective life is fairly close to a normal distribution. The engine manufacturer introduces an improvement into the manufacturing process for this component that increases the mean life to 5050 hours and decreases the standard deviation to 30 hours. Suppose that a random sample of components is selected from the “old” process and a random sample of components is selected from the “improved” process. What is the probability that the difference in the two sample means is at least 25 hours? Assume that the old and improved processes can be regarded as independent populations.
161 n
252 n
12 XX
Exercise 7-10◦ Suppose that the random variable has
the continuous uniform distribution
◦ Suppose that a random sample of observations is selected from this distribution. What is the approximate probability distribution of ? Find the mean and variance of this quantity.
Contents, figures, and exercises come from the textbook: Applied Statistics and Probability for Engineers, 5th Edition, by Douglas C. Montgomery, John Wiley & Sons, Inc., 2011.
X
otherwise0
101)(
xxf
12n
6X
General Concepts of Point General Concepts of Point EstimationEstimationBias of the estimator
is an unbiased estimator if
Minimum variance unbiased estimator (MVUE)◦ For all unbiased estimator of , the one
with the smallest variance is the MVUE for
◦ If , ,…, are from a normal distribution with mean and variance
)ˆ(E
)ˆ(E
X
1X nX2X 2
Standard error of an estimator
Estimated standard error◦ or or If is normal with mean and variance
and
Contents, figures, and exercises come from the textbook: Applied Statistics and Probability for Engineers, 5th Edition, by Douglas C. Montgomery, John Wiley & Sons, Inc., 2011.
)ˆ(ˆ
V
s )ˆ(se
X
n/2
nX
nS
X
Mean squared error of an estimate
Relative efficiency of to
Contents, figures, and exercises come from the textbook: Applied Statistics and Probability for Engineers, 5th Edition, by Douglas C. Montgomery, John Wiley & Sons, Inc., 2011.
2
22
22
2
2
)()ˆ(
))ˆ((]))ˆ(ˆ[(
))ˆ(()]ˆ(ˆ[2]))ˆ([(]))ˆ(ˆ[(
]))ˆ()ˆ(ˆ[(
])ˆ[()ˆ(MSE
biasV
EEE
EEEEEEE
EEE
E
2 1
)ˆ(MSE)ˆ(MSE
2
1
Example 7-4 Sample Mean and Variance Are Unbiased◦ Suppose that is a random variable with
mean and variance . Let , ,…., be a random sample of size from the population represented by . Show that the sample mean and sample variance are unbiased estimators of and , respectively.
Example 7-5 Thermal Conductivity◦ Ten measurements of thermal conductivity
were obtained:◦ 41.60, 41.48, 42.34, 41.95, 41.86◦ 42.18, 41.72, 42.26, 41.81, 42.04◦ Show that and
X 2 1X 2X nXn
X X2S 2
0898.010284.0ˆ
ns
x924.41x
Exercise 7-31◦ and are the sample mean and sample
variance from a population with mean and variance . Similarly, and are the sample mean and sample variance from a second independent population with mean and variance . The sample sizes are and , respectively.
◦ (a) Show that is an unbiased estimator of ◦ ◦ (b) Find the standard error of . How
could you estimate the standard error?◦ (c) Suppose that both populations have the same
variance; that is, . Show that
◦ Is an unbiased estimator of .
1X2
1S1 2
1
2X22S
2 22 1n
2n
21 XX 21
21 XX
222
21
2)1()1(
21
222
2112
nn
SnSnS p2
Moments◦Let , ,…, be a random sample
from the probability distribution , where can be a discrete probability mass function or a continuous probability density function. The th population moment (or distribution moment) is , = 1, 2,…. The corresponding th sample moment is = 1, 2, ….
Contents, figures, and exercises come from the textbook: Applied Statistics and Probability for Engineers, 5th Edition, by Douglas C. Montgomery, John Wiley & Sons, Inc., 2011.
Methods of Point Methods of Point EstimationEstimation
nX2X1X)(xf)(xf
k
kk)( kXE
k
n
ikiXn 1
1
Moment estimators◦Let , ,…, be a random
sample from either a probability mass function or a probability density function with unknown parameters , ,…, . The moment estimators , ,…, are found by equating the first population moments to the first sample moments and solving the resulting equations for the unknown parameters.
Contents, figures, and exercises come from the textbook: Applied Statistics and Probability for Engineers, 5th Edition, by Douglas C. Montgomery, John Wiley & Sons, Inc., 2011.
nX2X1X
m
1 2 m
21 mm
m
Maximum likelihood estimator◦Suppose that is a random variable
with probability distribution , where is a single unknown parameter. Let , ,…, be the observed values in a random sample of size . Then the likelihood function of the sample is
◦Note that the likelihood function is now a function of only the unknown parameter . The maximum likelihood estimator (MLE) of is the value of that maximizes the likelihood function .
X
);( xf 1x 2x nx
n
);();();()( 21 nxfxfxfL
)(L
Properties of a Maximum Likelihood Estimator◦Under very general and not restrictive
conditions, when the sample size is large and if is the maximum likelihood estimator of the parameter ,
◦(1) is an approximately unbiased estimator for
◦(2) the variance of is neatly as small as the variance that could be obtained with any other estimator, and
◦(3) has an approximate normal distribution.
Contents, figures, and exercises come from the textbook: Applied Statistics and Probability for Engineers, 5th Edition, by Douglas C. Montgomery, John Wiley & Sons, Inc., 2011.
n
])ˆ([ E
Invariance property◦Let , ,…., be the
maximum likelihood estimators of the parameters , , …, . Then the maximum likelihood estimator of any function of these parameters is the same function
◦of the estimators , ,…, .
Contents, figures, and exercises come from the textbook: Applied Statistics and Probability for Engineers, 5th Edition, by Douglas C. Montgomery, John Wiley & Sons, Inc., 2011.
1 2 k
1 2
k),...,,( 21 kh
)ˆ,...,ˆ,ˆ( 21 kh
1 2 k
Bayesian estimation of parameters◦Sample , ,…, ◦Joint probability distribution
◦Prior distribution for
◦Posterior distribution for
◦Marginal distribution
)()|,...,,(),,...,,( 2121 fxxxfxxxf nn
nX2X1X
)(f
),...,,(),,...,,(),...,,|(
21
2121
n
nn xxxf
xxxfxxxf
continuous ),,...,,(
discrete ),,...,,(),...,,(
21
21
21θdxxxf
θxxxfxxxf
n
n
n
Example 7-6 Exponential Distribution Moment Estimator◦ Suppose that , ,…, is a random sample
from an exponential distribution with parameter . For the exponential, .
◦ Then results in .Example 7-7 Normal Distribution
Moment Estimators◦ Suppose that , ,…, is a random sample
from a normal distribution with parameters and . For the normal distribution, and
◦ . Equating to and to gives
◦ and◦ Solve these equations.
nX2X1X
)(XEXXE )( X/1 X/1ˆ
nX2X1X 2
)(XE222 )( XE )(XE X )( 2XE
n
i in X1
21
X
n
i in X1
2122
Example 7-8 Gamma Distribution Moment Estimators◦ Suppose that , ,…, is a random
sample from a gamma distribution with parameters and , For the gamma distribution, and
◦ . Solve◦ and
Contents, figures, and exercises come from the textbook: Applied Statistics and Probability for Engineers, 5th Edition, by Douglas C. Montgomery, John Wiley & Sons, Inc., 2011.
nX2X1Xr
/)( rXE 22 /)1()( rrXE
Xr /
n
i in Xrr1
212/)1(
Example 7-9 Bernoulli Distribution MLE◦ Let be a Bernoulli random variable. The
probability mass function is
◦ where is the parameter to be estimated. The likelihood function of a random sample of size is
◦ Find that maximizes .
Contents, figures, and exercises come from the textbook: Applied Statistics and Probability for Engineers, 5th Edition, by Douglas C. Montgomery, John Wiley & Sons, Inc., 2011.
X
otherwise01,0)1(
);(1 xpp
pxfxx
pn
n
ii
n
ii
ii
xnxn
i
xx pppppL 11 )1()1()(1
1
p )( pL
Example 7-10 Normal Distribution MLE◦ Let be normally distributed with unknown
and known variance . The likelihood function of a random sample of size , say , ,…, , is
◦ Find .Example 7-11 Exponential Distribution
MLE◦ Let be exponentially distributed with
parameter . The likelihood function of a random sample of size , say , ,…, , is
◦ Find .
X 2
n 1X 2X nX
n
ii
i
x
nx
n
i
eeL 1
2222
)(2
1
2/2)2/()(
1 )2(1
21)(
X n
1X 2X nX
n
ii
ix
nn
i
x eeL 1
1
)(
Example 7-12 Normal Distribution MLEs for and◦ Let be normally distributed with mean
and variance , where both and are unknown. The likelihood function of a random sample of size is
◦ Find and .Example 7-13
◦ From Example 7-12, to obtain the maximum likelihood estimator of the function
◦ Substitute the estimators and into the function , which yields
X 2
n
n
ii
i
x
nx
n
i
eeL 1
2222
)(2
1
2/2)2/()(
1
2
)2(1
21),(
2
2
2
22 ),(h 2
2/1
1
212 )(ˆˆ
n
iin XX
Example 7-14 Uniform Distribution MLE◦ Let be uniformly distributed on the
interval 0 to . Since the density function is for and zeros otherwise, the likelihood function of a random sample of size is
◦ for , ,…,◦ Find .
Contents, figures, and exercises come from the textbook: Applied Statistics and Probability for Engineers, 5th Edition, by Douglas C. Montgomery, John Wiley & Sons, Inc., 2011.
X aaxf /1)( ax0
n
n
inaa
aL1
11)(
ax 10 ax 20 axn 0a
Example 7-15 Gamma Distribution MLE◦ Let , ,…, be a random sample from
the gamma distribution. The log of likelihood function is
◦ Find that
Contents, figures, and exercises come from the textbook: Applied Statistics and Probability for Engineers, 5th Edition, by Douglas C. Montgomery, John Wiley & Sons, Inc., 2011.
nX2X1X
n
ii
n
ii
n
i
xri
r
xrnxrnr
rexrL
i
11
1
1
)](ln[)ln()1()ln(
)(ln),(ln
xrˆ
)ˆ()ˆ(')ln()ˆln(
1 rrnxn
n
ii
Example 7-16 Bayes Estimator for the Mean of a Normal Distribution◦ Let , ,…, be a random sample from
the normal distribution with mean and variance , where is unknown and is known. Assume that the prior distribution for is normal with mean and variance ; that is,
◦ The joint probability distribution of the sample is
Contents, figures, and exercises come from the textbook: Applied Statistics and Probability for Engineers, 5th Edition, by Douglas C. Montgomery, John Wiley & Sons, Inc., 2011.
nX2X1X 2
2 0
20
)2/()(
0
20
20
21)(
ef
n
iix
nn exxxf 1
22 )()2/1(
2/221 )2(1)|,...,,(
◦ Show that
◦ Then the Bayes estimate of is
Contents, figures, and exercises come from the textbook: Applied Statistics and Probability for Engineers, 5th Edition, by Douglas C. Montgomery, John Wiley & Sons, Inc., 2011.
2
/)/(
)/
11)(2/1(
21
220
022
022
0),...,,|(
nnx
nn exxxf
nnx/
)/(22
0
022
0
Exercise 7-42◦ Let , ,…, be uniformly distributed on
the interval 0 to . Recall that the maximum likelihood estimator of is .
◦ (a) Argue intuitively why cannot be an unbiased estimator for .
◦ (b) Suppose that . Is it reasonable that consistently underestimates ? Show that the bias in the estimator approaches zero as gets large.
◦ (c) Propose an unbiased estimator for .◦ (d) Let . Use the fact that if
and only if each to derive the cumulative distribution function of . Then show that the probability density function of is
nX2X1Xaa )max(ˆ iXa
aa
)1/()ˆ( nnaaEa a
n
a
)max( iXY yY yX i
YY
◦ Use this result to show that the maximum likelihood estimator for is biased.
◦ (e) We have two unbiased estimators for : the moment estimator and
◦ , where is the largest observation in a random sample of size . It can be shown that and that
◦ . Show that if , is a better estimator than . In what sense is it a better estimator of ?
Contents, figures, and exercises come from the textbook: Applied Statistics and Probability for Engineers, 5th Edition, by Douglas C. Montgomery, John Wiley & Sons, Inc., 2011.
otherwiae0
0)(1
ayany
yf n
n
aa
Xa 2ˆ1 )max(]/)1[(ˆ2 iXnna )max( iX
n)3/()ˆ( 2
1 naaV )]2(/[)ˆ( 2
2 nnaaV 1n 2a1a
a
Exercise 7-50◦ The time between failures of a machine
has an exponential distribution with parameter . Suppose that the prior distribution for is exponential with mean 100 hours. Two machines are observed, and the average time between failures is hours.
◦ (a) Find the Bayes estimate for .◦ (b) What proportion of the machine do you
think will fail before 1000 hours?
Contents, figures, and exercises come from the textbook: Applied Statistics and Probability for Engineers, 5th Edition, by Douglas C. Montgomery, John Wiley & Sons, Inc., 2011.
1125x
Contents, figures, and exercises come from the textbook: Applied Statistics and Probability for Engineers, 5th Edition, by Douglas C. Montgomery, John Wiley & Sons, Inc., 2011.
top related