AMS572

AMS570.01 Final Exam Spring 2018

Name: _______________________________________________ ID: __________________________________ Signature: ________________________

Instruction: This is a close book exam. You are allowed two pages 8x11 formula sheet (2-sided). No cellphone or calculator or computer or smart watch is allowed. Cheating shall result in a course grade of F. Please provide complete solutions for full credit. The exam goes from 11:15am-1:45pm. Good luck!

1. Let be iid Poisson, let be a distribution, the conjugate family for the Poisson. (a) Find the posterior distribution of , and subsequently the Bayesian estimator of .

(b) Derive the mode of the posterior distribution of , and then discuss when and how we can derive the 100(1-α)% Bayesian HPD credible set for .

Solution:

(a)

The Poisson pdf is:

Therefore the likelihood function is:

By the factorization theorem, is a sufficient statistic for .

The mgf of Y|λ is:

Therefore we know Y|λ ~ Poisson.

I. For the prior:

The marginal pdf of Y is

Thus the posterior distribution is,

The Bayesian estimator for λ is:

(b)

Therefore when , the mode exists, we can obtain the Bayesian HPD [a, b] as illustrated below by the Theorem of Optimal Interval.

II. For the prior:

The posterior distribution is,

The Bayesian estimator for λ is:

The mode for the posterior of λ is:

2. Let be a random sample from an exponential distribution with pdf:

,

Please derive

(a) The maximum likelihood estimator for θ .

(b) Is the above MLE unbiased for θ ?

(c) Please derive the distribution of the first order statistic and further show whether is an unbiased estimator of θ or not.

(d) Please calculate the MSE (mean squared error) of Y, and the MSE of the MLE for θ , which one is smaller?

(e) Is there a best estimator (UMVUE) for θ? Please show the entire derivation.

(f) Is there an efficient estimator for θ? Please show the entire derivation.

Hint: Cramér-Rao Inequality: Let be unbiased for, where is a random sample from a population with pdf satisfying all regularity conditions. Then

Solution:

(a) The likelihood function is

The log likelihood function is

Solving

We obtain the MLE for

(b) Since

We know the MLE is an unbiased estimator for

(c) Now we derive the general formula for the pdf of the first order statistic as follows:

Therefore we have

Differentiating with respect to x, and then multiplying by (-1) at both sides leads to:

Now we can derive the pdf of the first order statistic for a random sample from the given exponential family. First, the population cdf is:

Now plugging in the population pdf and cdf we have:

when . Thus we know that , and its mean should be

Therefore we have:

That is, is an unbiased estimator of θ.

(d) Since both estimators are unbiased, MSE = Variance. We can easily derive the population variance for to be:

Thus we have:

Similar we know

Thus we have:

It is obvious that the variance, and thus the MSE, of the MLE is smaller.

(e) The exponential family with pdf:

is a regular exponential family with and therefore

is a complete sufficient statistic for . Thus the MLE of

is a function of the complete sufficient statistic, and is unbiased for . By the Lehmann Scheffe Theorem, the MLE is a best estimator (UMVUE) for

(f) Now we calculate the Cramer-Lower bound for the variance of an unbiased estimator for

Thus the Cramer-Rao lower bound is:

Therefore we claim that the MLE is an efficient estimator for .

***Note: You can also first prove (f) the MLE is an efficient estimator for , and subsequently immediately claim it is the (e) best estimator (UMVUE) for

3. To test whether the birth rates of boys and girls are equal, a random sample of 300 families with exactly three children within the age of 18 was taken and the distribution of the children’s gender is provided below. Please test whether the birth rates are equal or not at the significance level of α = 0.05. Please first derive your test using the pivotal quantity approach, including all steps for full points.

3 girls

2 girls, 1 boy

1 girl, 2 boys

3 boys

No. of families

36

114

117

33

Solution:

This problem can be done in several ways including, a large sample Z-test for one population proportion, or a Chi-square goodness of fit test. Here we simply use the large sample Z-test.

Part 1. Derivation of the general large sample Z-test for one population proportion.

Sampling from the Bernoulli Distribution

Toss a coin, and get the result as following: Head(H), H, Tail(T), T, T, H, …

Let A proportion of p is head, in the population.

, =0, 1; i = 1, 2, …, n.

(*Binomial distribution with n = 1)

Inference on p – the population proportion:

1 Point estimator:

, is the sample proportion and also, the sample mean

2 Large sample inference on p:

3 Large sample (the original) pivotal quantity for the inference on p.

Alternative P.Q.

*** You can use either of the pivotal quantity. In the following, we will use the first pivotal quantity, and illustrate the procedure for a two-sided test.

4 Large Sample Test:

Test statistic (large sample – using the original P.Q.):

At the significance level , we reject in favor of if:

Reject if

Part 2. Now we apply the above general test to the given problem.

Here we have . Let Y be the total number of girls among the 900 children, we have . Let p be the proportion of entrepreneurs with domestic cars, we have , and we are testing:

versus

The test statistics is:

Since , we can not reject the null hypothesis of equal birth rates at the significance level of 0.05.

4. Suppose we have a simple random sample from a normal population as follows: , where are both unknown.

(a) At the significance level α, please construct a test using the pivotal quantity approach to test versus where is a positive constant (*Please include the derivation of the pivotal quantity, the proof of its distribution, and the derivation of the rejection region for full credit.)

(b) At the significance level α, please derive the likelihood ratio test for testing versus . Subsequently, please show whether this test is equivalent to the one derived in part (a).

(c) Is the test in part (b) a uniformly most powerful test? Please provide detailed justifications for your answer.

Solution:

Note: Now that we are not just dealing with the two-sided hypothesis, it is important to know that the more general definition of is the set of all unknown parameter values under , while is the set of all unknown parameter values under the union of and .

Therefore we have:

So,

Reject in favor of if

Furthermore, it is easy to show that for each , the likelihood ratio test of versus rejects the null hypothesis at the significance level α for

By the Neyman-Pearson Lemma, the likelihood ratio test is also the most powerful test.

Now since for each , the most powerful size α test of versus rejects the null hypothesis for . This rejection region does not depend on -- it is the same rejection region, hence test, for each . Since this same test is most powerful for each , this test is UMP for versus , by the definition of the UMP test.

5. Suppose that the random variables satisfy , where are fixed constants, and are independent Gaussian random variables with mean 0 and variance , respectively ( are known constants, while is also a known constant). Please derive:

(a) The (ordinary) least squares estimator (LSE) for.

(b) The maximum likelihood estimator (MLE) for. Is the MLE the same as the LSE?

(c) At the significance level α, please derive a test for versus

Solution:

(a) To minimize

:

is the least squares regression line through the origin.

(b) Let

we have

Since are independent to each other, the likelihood is:

The log likelihood is:

Take derivative with respect to , and set it to zero, we obtain the MLE:

Therefore we can easily see that the MLE and the LSE are no longer identical when the are not all equal to each other.

(c) Let

Then

Furthermore, they are independent to each other.

We have the moment generating function for :

Therefore we found:

The pivotal quantity for is:

For the test of: versus ,

The test statistic is:

At the significance level , we reject in favor of if:

Reject if

*** That’s all, folks! ***

6

(

)

12

ˆ

,,,

n

hXXX

q

=

L

q

,1,,,

i

Xin

=

L

(

)

;

X

fx

q

(

)

(

)

(

)

1

1

2

2

2

ln;ln;

ˆ

XX

fxfx

VarnEnE

qq

q

qq

-

-

ìü

éù

ìü

éù

¶¶

æö

ïïïï

êú

³=-

íýíý

êú

ç÷

¶¶

êú

ïï

èø

ëû

ïï

îþ

ëû

îþ

1,.

0,.

th

i

th

iftheitossishead

X

iftheitossistail

ì

ï

=

í

ï

î

~()

i

XBernoullip

Þ

1

()(1)

ii

xx

ii

PXxpp

-

==-

i

x

1

ˆ

n

i

i

X

X

p

nn

=

==

å

ˆ

p

p̂ !~ N ( p,p(1− p)n

)

Z = p̂− pp(1− p)n

!~ N (0,1)

00

0

:

:

a

Hpp

Hpp

=

ì

í

¹

î

Z0 =p̂− p0p0 (1− p0 )n

!~H0N (0,1)

0

H

)

|

(

)

|

(

)

|

|

(|

0

0

0

0

0

0

H

c

Z

P

H

c

Z

P

H

c

Z

P

-

£

+

³

=

³

=

a

)

|

(

2

0

0

H

c

Z

P

³

×

=

)

|

(

2

0

0

H

c

Z

P

³

=

a

2

a

Z

c

=

\

0

H

2

0

|

|

a

Z

Z

³

0

:the distribution of the number of daugh

ters born follows a binomial distributio

n0.5

Hwithp

=

:the distribution of the number of daugh

ters born follows a binomial distributio

n0.5

a

Hwithp

¹

)

|

(

)

|

(

)

|

|

(|

0

0

0

0

0

0

H

c

Z

P

H

c

Z

P

H

c

Z

P

-

£

+

³

=

³

=

a

)

|

(

2

0

0

H

c

Z

P

³

×

=

)

|

(

2

0

0

H

c

Z

P

³

=

a

2

a

Z

c

=

\

0

H

2

0

|

|

a

Z

Z

³