5. estimation 5.3 estimation of the mean k. desch – statistical methods of data analysis ss10 is...

5. Estimation 5.3 Estimation of the mean

K. Desch – Statistical methods of data analysis SS10

Is an efficient estimator for μ ? depends on the distribution of x !

• there is an absolute lower limit on the variance of an estimator (Likelihood method – later)

• For a Gauss distribution is the most efficient estimator

• Alternative estimators:

Truncated mean: discard the (1-2r)n/2 largest and smallest elements of the sample when calculating the mean

r=0.5: sample mean, r→0: median. Example: Breit-Wigner (“μ“=0, Г=1)

ixn

1truncated mean r=0.2, discard 30% of high and low

x

x

5. Estimation 5.3 Estimation of the mean


Min-Max estimator

most efficient estimator uniformly distributed data

2

xxx minmax

ixn

12

xx minmax

but not robust (a single „wrong“ measurement can have huge impact)

5. Estimation 5.4 Estimation of the variance


The “sample variance”:

is an unbiased estimator of the true variance σ2

2

i2 )x(x

1n

1s

)-x)(2(x)-x()(x1n

1)-x()(x

1n

1s i

22i

2

i2

22i )x()(x

1n

1

2ii )x(2)x)(x2n()x()x(2)x)(x(2 as:

expectation value of s2: 2 2 2i

1E[s ] E (x - ) -E (x - )

n -1

22

2 σ1n

1n

n

σnσn

1n

1

2

2i2 2 2

i i2 2

x 1 1 σE (x ) E E ( x n ) E ( (x )

n n n nas:

5. Estimation 5.4 Estimation of the variance


Why ?

Since true mean μ is not known and must be estimated also from sample, one looses one degree of freedom

If the true mean would be known, then

would be an unbiased estimator of the variance

Estimator for Variance of s2 :

1n

1

2

i2 )x(

n

1S

42 2s

V[s ]n 1

6. Max. Likelihood 6.1 Maximum likelihood method


(x1,…xn) is a data sample with a probability density f(xi,a).

How can one construct an estimator for a when f is known (analytically or numerically)?

Likelihood-function: a measure for the probability to obtain (x1,…xn) for fixed a:

Regard L as a function of a ! (xi are fixed, L is not a p.d.f.)

Principle of maximum likelihood (ML)

Best estimator for a, , is the one which maximizes the Likelihood function:

Important: f(x;a) has to be correctly normalized for all a:

n

iin ;a)f(x;a)f(x...;a)f(x;a)f(xL(a)

121

a

ˆa aL(a) maximum

1f(x;a)dx



When sample size n is large, product is impractical (numerical problems)

use the logarithm of the Likelihood function intstead

product in L is transformed into a sum:

0)ˆ(

);(lnln)(1

a

alMaximumaxfL(a)al

n

ii

More than one parameter:

Often negative log-Likelihood is used minimize F(a)

(or even -2lnL → consistency with χ2 estimater, later)

Properties of ML estimators

1) ML estimators are usually consistent

2) ML estimators are (only) asymptotically unbiased

3) ML estimators are efficient! (they reach the limit of minimum variance)

4) ML estimators are invariant under parameter transformation

estimator for θ(a) ? When is a ML estimator for a, then is the ML estimator for . :

Drawback: ML estimators do not test whether the data agrees with f – separate tests are necessary (no “goodness of fit” measure)



LalF(a) ln)(

1 mk

k

l(a ,...,a )a (k 1,...,m) 0

a

)ˆ(ˆ||| ˆ)ˆ(ˆ

aa

L

a

L

aaa

a ˆ ˆ(a)

Example of ML estimators : anglular distribution

measurements xi, i=1,n



)1(2

1; axa)f(x cosx

)1(2

1ln)(ln

1i

n

i

axaLF(a)



2

2

2

)(

2

1),;(

x

exf

Example of ML estimators: μ and σ2 of a Gaussian

n

iixfLF

1

222 ),;(log),(log),(

n

i

ix

12

2

2 2

)(1log

2

1

2

1log

Estimator for μ :

n

iix

n

xL

12ˆ

2 1ˆ

)ˆ(|

),(log0

4

2

22

2

ˆ2

)ˆ(

ˆ

1

2

1|

),(log0

22

ixLEstimator for σ2 :

22

2

2

)ˆ(1

0)ˆ(

i

i xn

xn

ML estimator of σ2 is biased ! ( Unbiased: )

22 )ˆ(1

1 ixn

s

Example of ML estimators : exponential distribution

p.d.f. :

Construct ML estimator for parameter :

Log- Likelihood function:

Maximum:



/1

);( tetf

n

i

in

ii

ttfL

11

1log);(log)(log

0ˆ

)ˆ(log

L

0ˆ

1)(log

12

i

n

i

i tn

tL

arithmetic mean

n

iitn 1

1

Expectation of :



nnn dtdttftfttE ...);(...);(),...,(ˆ...]ˆ[ 111

n

tt

i dtdteetn

n

...1

...11

... 1

1

n

i ijj

t

i

t

i dtedtetn

ji

1 0

111

n

in 1

1is unbiased

“Statistical error of parameter “

depends on true value of , which we don’t know. How one can estimate ? → Parameter transformation of ML estimator

( , )

Variance of :

How efficient is the estimator ?

Rarely possible analytically → Monte Carlo method



22 ]ˆ[]ˆ[]ˆ[ EEV

n

tt

i dtdteetn

n

...1

...11

... 1

21

1 n2t t

i 1 n

1 1 1... t e ... e dt ...dt

n

n

2

]ˆ[V]ˆ[V

)(a )ˆ(ˆ a

nV

2ˆ)](ˆ[

n

VV2ˆ

)ˆ](ˆ[]ˆ[

,ˆ

ˆ2

2ˆ n

n

ˆ

ˆ ˆ



Measurement : means that an experiment estimates the

0.7 ± 0.1

parameter to be 0.7 and by continuous repetition the shows a standard deviation of 0.1

ˆˆ

Rao-Cramer-Frechet (RCF) bound of minimum variance (w/o proof)

Variance of an estimator of single parameter is limited as:

is called “efficient” when the bound exactly archived

• When an efficient estimator exists, it is a ML estimator

• All ML estimators are efficient for n → ∞

Example: exponential distribution :

as (b=0) :



2

2

2

2

)(log

1][

L

E

b

V

ˆ2

112

1log

21

22

2 nt

n

nL n

ii

0b

nEnnE

V2

22

]ˆ[21

1

ˆ21

1]ˆ[

5. estimation 5.3 estimation of the mean k. desch – statistical methods of data analysis ss10 is...

Documents

estimator likelihood

variance estimator

maximum likelihood method

data sample

unbiased estimator

max estimator

best estimator

gaussian estimator