emma woolliams presentation to: orm club meeting 28 -...

Making sense of uncertainties

Emma Woolliams

Presentation to: ORM Club Meeting28th June 2006

Why do we do uncertainty budgets?

• We have to• Makes results meaningful• Quality• Manufacturing tolerances• Improve our measurements

Tolerance

Range of acceptable measurements

Why do we fear doing them?

∑i

i

yx∂∂

2 2A1,1 Ar1

2 2A1,2 Ar2

2 2Ar

2 2B3,1 Br1

2 2B3,2 Br2

2 2B

1 A2,12 2Ar2 A2,2

r1 B4,12Br2 B4,

0 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0

0 00 0

0 0

0 00 0 0

0 0 0 0 0 0 0 0 0 0 0 00 0 0

0

0

0

0 00 0

e

e

e

e

e

e

e

e

u uu u

u

u uu u

u uu u

uu u

=V 2 2 2 2p1,1 pr1 pr1 pr1

2 2 2 2p1,2 pr2 pr2 pr2

2 2 2 2pr1 p2,1 pr1 pr1

2 2 2 2pr2 p2,2 pr2 pr2

2 2 2 2pr1 pr1 p3,1 pr1

2 2

22

pr2 pr2

0 0 0 0 0 0 0 00 0 0 0 0 0 0 00 0 0 0 0 0 0 00 0 0 0 0 0 0 00 0 0 0

0 0 0 00 0 0 0

0 0 0 00 0 0 0

0 0 0 00 0

0 0 0 00 0 0 0 0 0 0 00 0 0 0 0 0 0 0

e

e

e

e

e

u u u uu u u u

u u u uu u u u

u u u uu u 2 2

p3,2 pr22 2 2 2pr1 pr1 pr1 p4,1

2 2 2 2pr2 pr2 pr2 p4,2

0 00 0 00 0 0 0 0 0 0 0

0 0 0 0 0 0 0 00

0 0 0 0

e

e

e

u uu u u u

u u u u

⎛ ⎞⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎝ ⎠

What’s really difficult?

∑

i

i

yx∂∂

The GUM

• Guide to the expression of uncertainty in measurement (GUM)• ISO 1995: ISBN number is 92-67-10188-9, 1995• (e.g. British Standard PD 6461-3:1995)

• Formalises and standardises determination of Measurement Uncertainty

Where do ideas come from?

Some concepts: Type A vs Type B

• Type A: Determined statistically, e.g.

• Type B: Determined from prior knowledge, e.g.

In range

More concepts: correlation

Measurements are correlated – uncertainties are

NOT0

0.2

0.4

0.6

0.8

1

1.2

400 420 440 460 480 500 520 540 560 580 60

0

0.2

0.4

0.6

0.8

1

1.2

400 420 440 460 480 500 520 540 560 580 60

The basic equation

( )1 2, , , Ny f x x x= K

( ) ( ) ( ) ( ) ( )1

2 2 2

1 1 12 ,

N N N

i i i j i j i ji i j i

u y c u x c c u x u x r x x−

= = = +

= +∑ ∑ ∑

ii

ycx∂

=∂

The steps to an uncertainty budget

( ) ( ) ( ) ( ) ( )1

2 2 2

1 1 12 ,

N N N



= = = +

= +∑ ∑ ∑

• Convert to Standard Uncertainties• Determine Sensitivity Coefficients• Determine Correlation Coefficients• Combine Budget

• Determine Degree of Freedom and Coverage Factor• Expanded Uncertainty Statement

( )1 2, , , Ny f x x x= K

• Formulate Measurement Model• Describe Measurement Process

Developing a model – part 1

2V VI E r=

2V,test test CCFI D C r=

2V,test test CCF cos cosI D C r θ φ=

2V,test test CCF uniformitycos cos ( ) ( , , )I D C r k T k rθ φ θ φ=

0

0.2

0.4

0.6

0.8

1

1.2

350 400 450 500 550 600 650 700 750wavelength

lux meter responsivityV(lambda)Tungsten at 2856 KBlue LED

Developing a model – part 2

• Listing sources of uncertainty, are they correlated?2

V,test test CCF uniformitycos cos ( ) ( , , )I D C r k T k rθ φ θ φ=

Display – Resolution, repeatability, calibration accuracy

CCF – calculated for this source or a batch, variation, luxmeter knowledge

Distance – Repeatability, calibration of ruler/instrument

Angles – offset angles possible

Temperature sensitivity

Uniformity of source at this distance – angular and spatialCorrelations

Consider – size of effects, how determined, distribution functions etc.


( ) ( ) ( ) ( ) ( )1

2 2 2

1 1 12 ,

N N N



= = = +

= +∑ ∑ ∑



( )1 2, , , Ny f x x x= K


Doing the maths – part 1 Sensitivity coefficients

2V,test test CCF uniformitycos cos ( ) ( , , )I D C r k T k rθ φ θ φ=

( ) ( ) ( ) ( ) ( )1

2 2

1 1 12 ,

N N N



= = = +

= +∑ ∑ ∑ ii

ycx∂

=∂

V,test V,test2CCF uniformity

test test

cos cos ( ) ( , , )I I

C r k T k rD D

θ φ θ φ∂

= =∂

Doing the maths – part 2 Convert to standard uncertainties

•Type A•Type B: Calibration Certificates(for these divide by k (often 2))

13

•Type B (most)•You know it’s in a range

Doing the maths – part 3 Combining the uncertainties

( ) ( ) ( ) ( ) ( )1

2 2

1 1 1

2 2 ,N N N

i i j i j i ji

ii j i

u y u xc c c u x u x r x x−

= = = +

= +∑ ∑ ∑ ∑

Value Equation uncertainty Source of uncertainty Value in units Relative value Probability distribution Divisor ci Relative Absolute20 Displayed value / lux Source drift 0.1 0.500% RECTANGULAR 1.732051 1 0.29% 4.789847

Noise 0.1 0.500% GAUSSIAN 1 1 0.50% 8.296258Calibration of lux meter 1.400% GAUSSIAN 95% 2 1 0.70% 11.61476

1.024229 Colour Correction Factor Spectral difference in test source 0.001832096 0.179% RECTANGULAR 1.732051 1 0.10% 1.7135739 Distance / m Systematic error in ruler 0.001 0.011% GAUSSIAN 1 2 0.02% 0.368723

Measurement error 0.02 0.222% RECTANGULAR 1.732051 2 0.26% 4.257641


( ) ( ) ( ) ( ) ( )1

2 2 2

1 1 12 ,

N N N



= = = +

= +∑ ∑ ∑



( )1 2, , , Ny f x x x= K


Expanded uncertainties

66%95%

With a Gaussian – multiply by 2

If not a Gaussian – multiply by a coverage factor k

Is the Type A uncertainty small?

Have you taken lots of measurements?

Yes

k = 2 No

Welch-Satterthwaite

Relative and absolute uncertainties

A x y= + 1, 1A Ax y

∂ ∂= =

∂ ∂( ) ( ) ( )2 2 2u A u x u y= +

A xy= ,A Ay xx y

∂ ∂= =

∂ ∂,A A A Ay

x x y y∂ ∂

= = =∂ ∂

( ) ( ) ( )22

2 2 2A Au A u x u yx y

⎛ ⎞⎛ ⎞= + ⎜ ⎟⎜ ⎟⎝ ⎠ ⎝ ⎠

( ) ( ) ( )2 2 2

2 2 2

u A u x u yA x y

= +

Example 1: The average of two measurements

• Two measurements made with two different (but highly correlated) references – what is the uncertainty?

[ ]1 212

E E E= +

( ) ( ) ( ) ( ) ( ) ( )2 2

2 2 21 2 1 2 1 1

1 2 1 2

2 ,E E E Eu E u E u E r E E u E u EE E E E

⎛ ⎞ ⎛ ⎞ ⎛ ⎞⎛ ⎞∂ ∂ ∂ ∂= + +⎜ ⎟ ⎜ ⎟ ⎜ ⎟⎜ ⎟⎜ ⎟ ⎜ ⎟ ⎜ ⎟⎜ ⎟∂ ∂ ∂ ∂⎝ ⎠ ⎝ ⎠ ⎝ ⎠⎝ ⎠

( ) ( ) ( ) ( ) ( )1

2

1 1 1

2 2 ,N N N

i i j i j i ji

ii j i

u y u xc c c u x u x r x x−

= = = +

= +∑ ∑ ∑

Averaging two measurements

[ ]1 212

E E E= +

1

2

1

2 2

1

1

11

AA

A

E

E

E

E

EE

ε

ε

ε

ε=

==

= ±

′

=±

′

′ ≈

±

[ ]1 22AEE εε

′= + ( ) ( ) ( ) ( )2

2 2 2

2 2 2 2

21

1

E E Eu E u u uAA

εε

εε

⎛ ⎞ ⎛ ⎞ ⎛ ⎞∂ ∂ ∂= + +⎜ ⎟ ⎜ ⎟ ⎜ ⎟⎜ ⎟ ⎜ ⎟ ⎜ ⎟∂ ∂ ∂⎝ ⎠ ⎝ ⎠ ⎝ ⎠

( )( ) ( ) ( )

2 22

12 22 21

2

u Eu u u

EA εε⎛ ⎞ ⎡ ⎤= + +⎜ ⎟ ⎣ ⎦⎝ ⎠

( )( ) ( )

22 2

2

12

Eu

EA

uu ε⎛ ⎞= + ⎜ ⎟

⎝ ⎠

1

,1 2

E E E EA A

E Aε

′∂ ∂= = =

∂ ∂

Example 2: Measuring the illuminance of a lamp using a photometer


V,test V,test2CCF

test test

cos cosI I

C rD D

θ φ∂

= =∂

V,test V,test2test

CCF CCF

cos cosI I

D rC C

θ φ∂

= =∂

V,test V,testtest CCF cos cos2 2

I ID C r

r rθ φ

∂= =

∂

V,test 2test CCF sin cos ???

ID C r φθ

θ∂

= − =∂

Cosines


V,test 2test CCF sin cos ???

ID C r φθ

θ∂

= − =∂

sin 0 ( ) 0 ?????uθ θ= ∴ =

( ) ( ) ( ) ( ) ( ) ( )2 2 2 2

V,test V,test V,test V,test2 2 2 2 4V,test test

2V,t

CCF 2tes

est

t CC2

F

4 12

12

I I I Iu I u D u C u r u

D C rI

u θθ

φφ

∂ ∂ ∂⎛ ⎞ ⎛ ⎞ ⎛ ⎞= + + + +⎜ ⎟ ⎜ ⎟ ⎜ ⎟∂ ∂ ∂⎝ ⎠⎝ ⎠ ⎝ ⎠

A quick tour of some advanced topics: Monte Carlo

tesV, t C2

Fe Ct st co os scC rDI θ φ=

To conclude

( ) ( ) ( ) ( ) ( )1

2 2 2

1 1 12 ,

N N N



= = = +

= +∑ ∑ ∑



( )1 2, , , Ny f x x x= K


Conclusions

∑i

i

yx∂∂

2 2A1,1 Ar1

2 2A1,2 Ar2

2 2Ar

2 2B3,1 Br1

2 2B3,2 Br2

2 2B

1 A2,12 2Ar2 A2,2

r1 B4,12Br2 B4,

0 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0

0 00 0

0 0

0 00 0 0

0 0 0 0 0 0 0 0 0 0 0 00 0 0

0

0

0

0 00 0

e

e

e

e

e

e

e

e

u uu u

u

u uu u

u uu u

uu u

=V 2 2 2 2p1,1 pr1 pr1 pr1

2 2 2 2p1,2 pr2 pr2 pr2

2 2 2 2pr1 p2,1 pr1 pr1

2 2 2 2pr2 p2,2 pr2 pr2

2 2 2 2pr1 pr1 p3,1 pr1

2 2

22

pr2 pr2

0 0 0 0 0 0 0 00 0 0 0 0 0 0 00 0 0 0 0 0 0 00 0 0 0 0 0 0 00 0 0 0

0 0 0 00 0 0 0

0 0 0 00 0 0 0

0 0 0 00 0

0 0 0 00 0 0 0 0 0 0 00 0 0 0 0 0 0 0

e

e

e

e

e

u u u uu u u u

u u u uu u u u

u u u uu u 2 2

p3,2 pr22 2 2 2pr1 pr1 pr1 p4,1

2 2 2 2pr2 pr2 pr2 p4,2

0 00 0 00 0 0 0 0 0 0 0

0 0 0 0 0 0 0 00

0 0 0 0

e

e

e

u uu u u u

u u u u

⎛ ⎞⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎝ ⎠

emma woolliams presentation to: orm club meeting 28 -...

Documents