lecture 4: statistics review ii date: 9/5/02 hypothesis tests: power estimation: likelihood,...

Lecture 4: Statistics Review II

Date: 9/5/02Hypothesis tests: powerEstimation: likelihood, moment estimation, least squareStatistical properties of estimators

Types of Errors

False positive (Type I): Probability () that H0 rejected when it is true.

False negative (Type II): Probability () of accepting H0 when it is false.

Accept H0 Reject H0

H0 true 1- Type I = H0 false Type II = power = 1-

Example: Error

H0: Central Chi-SquareHA: Non-Central Chi-Square withnon-centrality parameter E(G)

1-

1-

Power of a Test

Definition: The statistical power of a test is defined as 1-

The power is only defined when HA is defined, the experimental conditions (e.g. sample size) are known and the significance level has been selected.

Example: calculate sample size needed to obtain particular linkage detection power.

Estimation

Hypothesis testing allows us to make qualitative conclusions regarding the suitability or not of a statement (H0).

Often we want to make quantitative inference, e.g. an actual estimate of the recombination fraction, not just evidence that genes are linked.

Estimation

Definition: Point estimation is the process of estimating a specific value of the parameter based on observed data X1, X2,…,Xn.

Definition: Interval estimation is the process of estimating upper and lower limits within which the unknown parameter occurs with certain probability.

Definition: The maximum likelihood estimator is the which maximizes the likelihood function.

The MLE is obtained by analytically solving the score, S=0 grid search Newton-Raphson iteration Expectation and Maximization (EM) algorithm

Maximum Likelihood Estimation

MLE: Grid Search

Plot likelihood L or log likelihood l vs. parameter throughout the parameter space.

Obtain MLE by visual inspection or search algorithm.

MLE: Grid Search Algorithm

1. Initially estimate 0. Pick a step size .

2. At step n, evaluate L (or l) on both sides of n.

3. Choose n+1= n+ if L is increasing to the right, else choose n+1= n-.

4. Repeat steps 2 and 3 until no longer advance. Choose smaller , and repeat steps 2, 3, and 4 until desired accuracy met.

MLE: Grid Search Problems

Multiple peaks result in a failure to find the global maximum likelihood.

Solving for multiple simultaneous parameters gets computationally intensive and difficult to interpret visually when there are more than two parameters.

Example: 2-Locus with Non-Penetrant Allele

Let be the recombinant fraction between marker A and gene of interest B.

Let be the probability that the allele of the gene of interest (f) fails to be detected at the phenotype level (i.e. 1- is the penetrance).

Cross +F/+F –f/–f. Score gametes of an F1 +F/–f individual for +/-

phenotype and P/p phenotype, where P means F or non-penetrant f and p means penetrant f.

B(F or f)A(+ or -)

Experimental Data

P(+F gamete) = P(–f gamete) = 0.5(1-) P(–F gamete) = P(+f gamete) = 0.5 P(+P gamete) = 0.5(1-) + 0.5 P(–P gamete) = 0.5 + 0.5(1-) P(+p gamete) = 0.5(1-) P(–p gamete) = 0.5(1-) (1-) Observe n+P, n-P, n+p, n-p.

Experimental Data: Log Likelihood

11log1log

1log1log

111

11

pp

PP

nn

nn

nn

nnl

Lpp

PP

Experimental Data: Grid Search

Newton-Raphson: One Parameter

Let S() be the score. Then the MLE is obtained through the

equation Taylor expansion of S() for n near the

MLE, gives

0)ˆ( S

0ˆˆ

d

dSSS m

mm

ddS

S

m

mm /

ˆ

ddS

S

m

mmm /1

Newton-Raphson: Analysis

NR fits a parabola to the likelihood function at the point of the current parameter estimate. Obtain a new parameter estimate at the maximum of the parabola.

NR may fail when there are multiple peaks. NR may fail when the information is zero (when the

estimate is at the extremes). Recommendations: Use multiple starting initial

values. Bound new estimates.

Newton-Raphson: Multiple Parameters

mmmm SIN

11

1

N is the total sample size.S(m) is the score vector evaluated at m.I-1 (m) is the inverse information matrix evaluated at m.

EM Algorithm: Incomplete Data

The notion of incomplete data:

AB ab Ab aB

AB AB/AB AB/ab AB/Ab AB/aB

ab ab/AB ab/ab ab/Ab ab/aB

Ab Ab/AB Ab/ab Ab/Ab Ab/aB

aB aB/AB aB/ab aB/Ab aB/aB

EM Algorithm: Example of Incomplete Data

+P gamete may result from nonrecombinant +F or from recombinant, non-penetrant +f.

+p gamete can only result from penetrant, nonrecombinant +f.

–P gamete can result from recombinant –F or from nonrecombinant, non-penetrant –f gene.

–p gamete can result only from nonrecombinant –f.

EM Algorithm

Make an initial guess 0. Expectation step: Pretend that n for iteration n is

true. Estimate the complete data. This usually request distribution of complete data conditional on the observed data. For example: P(recombinant|observed phenotype).

Maximization step: Compute the maximum likelihood estimate for step n .

Repeat E & M steps until likelihood converges.

n

Example: E Step

E Step:

5.015.0

5.0

P

ANDpenetrant -nonPpenetrant-nonP

penetrant-nonPpenetrants-nonE

trecombinanPtsrecombinanE

4

1

4

1

P

PP

Pf

Pf

iii

iii

Example: M Step

M Step:

recessive

1

1

penetrants-nonE

tsrecombinanE

N

N

n

n

Moment Estimation

Obtain equations for the population moments in terms of the parameters to estimate.

Substitute the sample moments and solve for the parameters.

For example: binomial distribution

m1 = np

n

X

n

mp

n

ii

11ˆ

Example: Moment Estimation

ntP

tp

tPpppP

PpP epepeppttt ,,mgf

pp

PP

pp

PP

npm

npm

npm

npm

nfp

nfp

nfp

nfp

pp

PP

pp

PP

ˆ

ˆ

ˆ

ˆ

15.0ˆ

115.0ˆ

15.0ˆ

15.0ˆ

P

p

p

P

p

p

p

p

Moment Estimation: Problems

Large sample properties of ML estimators are usually better than those for the corresponding moment estimators.

Sometimes solution of moments equations are not unique.

Least Squares Estimation

XY

XXYXYY '''2''

YXXX ''ˆ 1

YXYY

YXYY

RRreduced

full

SSE

SSE

''ˆ'

'ˆ'

kNkNF

kN

SSEkN

SSE

Ffull

reduced

,1~1

Variance of an Estimator

k

iik 1

22ˆ

ˆ1

1ˆ

•Suppose k independent estimates are available for :

•Suppose you have a large sample, then the variance of the MLEis approximately:

nI

1ˆ 2

ˆ Cramer-Raolower bound for variance

•Empirical estimates using resampling techniques.

Variance: Linear Estimator

k

i

k

ijjiji

k

iiikk ccccc

1 11

211 ,Cov2VarVar

Variance: General Function f(, , …, )

k

i

k

i

k

ijji

jii

i

k

d

df

d

df

d

df

f

1 1 1

2

1

,Cov2Var

,,Var

Bias

The mean square estimator (MSE) is defined as

bias

2ˆE MSE

22

ˆˆE

ˆEˆEˆE

MSE

If an estimator is unbiased, the MSE and variance are the same.

Estimating Bias

Bootstrap:

b

iiB b

Bias1

ˆˆ1

bootstrap estimator for bootstrap trial i

original estimate

Confidence Interval

Because of sampling error, the estimate is not exactly the true parameter value .

A confidence interval is symmetric if

A confidence interval is non-symmetric if

A confidence interval is one-sided if

ULˆP

0P OR 0P ˆˆ UL

UL ˆˆ PP

2PP ˆˆ UL

Confidence Interval: Normal Approximation I

Need pivotal quantity, i.e. a quantity that depends on the data and the parameters but whose distribution does not.

If the estimate is unbiased and normally distributed with variance , then the pivotal quantity is

ˆ

ˆ

ˆ

Confidence Interval: Normal Approximation II

The MLE is asymptotically normally distributed.

ˆ15.0ˆ15.0

15.0ˆ

15.0

ˆˆP

ˆP

zz

zz

Confidence Interval: Nonparametric Approximation

xxCDF b P

5.0,15.0 11 CDFCDF

percentile method

Bootstrap Example

Generate a multinomial random variable with the given proportions b times and generate a bootstrap dataset. Estimate parameters and .

+P +p –P –p

Count 168 3 52 163

Proportion 0.44 0.01 0.13 0.42

b b

Confidence Interval: Likelihood Approach

Let Lmax be the maximum likelihood for a given model. Find the parameter values L and U such that

log Lmax – log L() = 2

Then (L, U) serves as a confidence interval.

LOD Score Support

The LOD score support for a confidence interval is

log10Lmax –log10L

where L is the likelihood at the limit values of the parameter.

In practice, you plot the LOD score support for various values of the parameter and choose the upper and lower bounds such that the LOD score support is 1.

Choosing Good Confidence Intervals

The actual coverage probability should be close to the confidence coefficient.

Should be biologically relevant. For example, the following is not:

(0.1,0.6)

Good Estimator: Consistent

An estimator is mean squared error consistent if the MSE approaches zero as the sample size approaches infinity.

An estimator is simple consistent if 1ˆPlim

n

Good Estimator: Unbiased

An unbiased estimator is usually better than a biased one, but this may not always be true. If the variance is larger, what have we gained?

There are bootstrap techniques for obtaining a bias-corrected estimate. These are computationally more intensive than bootstrap, but sometimes worth it.

Good Estimator: Asymptotically Normal

If the pivotal quantity

is normal with mean 0 and variance 1 as the sample size goes to infinity, it can be a very convenient property of the estimator.`

ˆ

ˆ

Good Estimator: Confidence Interval

A good estimator should have a good way to obtain an confidence interval. MLE are good in this way if the sample size is large enough.

Sample Size for Power

1-

1-

Need E(G)> 12

,,1 2

df

E(G)=nE(Gunit)

unit

12

,,1

E

2

Gn df

Sample Size for Target Confidence Interval

Confidence interval by normal approximation is

The bigger the range , the less precise the confidence interval.

Suppose we wish to have

ˆ2/1ˆ

z

ˆ2/12 z

dz ˆ2/12

Sample Size for Target Confidence Interval II

Then,

I

d

z

n

dnI

z

2

2/1

2/1

2

12

Summary

Distributions Likelihood and Maximum Likelihood

Estimation Hypothesis Tests Confidence Intervals Comparison of estimators Sample size calculations