Slide 1

GEODETIC INSTITUTE LEIBNIZ UNIVERSITY OF HANNOVER GERMANY Ingo Neumann and Hansjrg Kutterer The probability of type I and type II errors in imprecise hypothesis testing REC 2008 Reliable Engineering Computing February 20, 2008

Slide 2

Motivation 2 February 20, 2008 The probability of type I and type II errors Sometimes the uncertainty budget in geodetic applications is too optimistic Examples: A geometric leveling around a big lake in Germany, Switzerland and Austria Control networks that are observed with two different techniques (i) terrestrial measurements (ii) satellite measurements Why? Ignorance of non-stochastic errors in the measurements and in the preprocessing steps of the measurements!?

Slide 3

Motivation 3 February 20, 2008 The probability of type I and type II errors

Slide 4

Motivation 4 February 20, 2008 The probability of type I and type II errors The model constants are only partially representative for the given situation (e. g., the model constants for the refraction index for distance measurements). The number of additional information (measurements) may be too small to estimate reliable distributions. Displayed measurement results are affected by rounding errors. Other non-stochastic errors of the reduced observations occur due to neglected correction and reduction steps and for effects that cannot be modeled. Why non-stochastic errors?

Slide 5

Motivation Uncertainty modeling in geodetic data analysis Statistical hypothesis tests in case of imprecise data General form of a linear hypothesis Probability of type I/II errors Geodetic applications One and multidimensional case (weak imprecision) Congruence test (strong imprecision) Conclusions and future work 5 February 20, 2008 The probability of type I and type II errors Agenda

Slide 6

Uncertainty Modeling 6 February 20, 2008 The probability of type I and type II errors Systematic effects Measurement process: -Stochasticity -Observation imprecision -(Outliers) Object fuzziness, etc... Stochastics (Bayesian approach) Interval mathematics Fuzzy theory Occurring uncertainties in this presentation :

Slide 7

Requirements: Adequate description of Stochastics Adequate description of Imprecision Solution: Describing the influence factors for the preprocessing step of the originary observation with fuzzy sets e. g., LR-fuzzy-number 7 February 20, 2008 The probability of type I and type II errors Uncertainty Modeling

Slide 8

Sensitivity analysis for the calculation of the imprecision of some parameters x of interest: - Instrumental error sources - Uncertainties in reduction and corrections - Rounding errors Influence factors (p) Linearization Partial derivatives for all influence factors Imprecision of the influence factors 8 February 20, 2008 The probability of type I and type II errors Uncertainty Modeling

Slide 9

9 February 20, 2008 The probability of type I and type II errors Uncertainty Modeling -discitization Imprecise analysis ( ): Observation Imprecision Stochastic (Bayesian approach)

Slide 10

Parameter Estimation (Linear case) Global test Outlier detection Congruence tests Model selection Sensitivity analysis These tests are based on a linear hypothesis Statistical hypothesis tests Tasks in (linear) parameter estimation: 10 February 20, 2008 The probability of type I and type II errors

Slide 11

General form of a linear hypothesis: Statistical hypothesis tests n:= number of observations u:= number of parameters precise case Introduction of a linear hypothesis: 11 February 20, 2008 The probability of type I and type II errors

Slide 12

Introduction of a quadratic form / test value: Test decision: And with imprecision? 12 February 20, 2008 The probability of type I and type II errors General form of a linear hypothesis: Statistical hypothesis tests

Slide 13

13 February 20, 2008 The probability of type I and type II errors General form of a linear hypothesis: Statistical hypothesis tests With imprecise influence factors p Introduction of a quadratic form / test value:

Slide 14

14 February 20, 2008 The probability of type I and type II errors General form of a linear hypothesis: Statistical hypothesis tests Resulting test scenario 1D comparison Final decision based on the comparison of the tests value with the regions of acceptance and rejection (card criterion)

Slide 15

Degree of agreement Degree of disagreement Basic idea 15 February 20, 2008 The probability of type I and type II errors Statistical hypothesis tests With:

Slide 16

Degree of rejectability Test decision: Degree of agreement Degree of disagreement 16 February 20, 2008 The probability of type I and type II errors Statistical hypothesis tests

Slide 17

The card criterion: with: ~~~ Overlap region 17 February 20, 2008 The probability of type I and type II errors with: Statistical hypothesis tests

Slide 18

With denoting the inverse function Probability of a type I error in the imprecise case: Statistical hypothesis tests.. 18 February 20, 2008 The probability of type I and type II errors

Slide 19

(3) Find in such a way that the following Equation is fulfilled within a negligible threshold: Probability of a type I error in the imprecise case: Statistical hypothesis tests.. (1) Choose an adequate value for : (2) Compute. 0.9 19 February 20, 2008 The probability of type I and type II errors

Slide 20

Probability of a type II error in the imprecise case: Statistical hypothesis tests.. 20 February 20, 2008 The probability of type I and type II errors Choose the non-centrality parameter or the probability of a type II error in the imprecise case

Slide 21

(3) Find in such a way that the following Equation is fulfilled within a negligible threshold: Probability of a type II error in the imprecise case: Statistical hypothesis tests.. (2) Choose an adequate value for : 21 February 20, 2008 The probability of type I and type II errors (1) Compute the probability of a type I error in the im- precise case

Slide 22

(3) Find in such a way that the following Equation is fulfilled within a negligible threshold: Non-centrality parameter in the imprecise case: Statistical hypothesis tests.. (2) Choose an adequate value for : 22 February 20, 2008 The probability of type I and type II errors (1) Compute the probability of a type I error in the im- precise case

Slide 23

Geodetic applications.. 23 February 20, 2008 The probability of type I and type II errors A geodetic monitoring network of a lock: The lock Uelzen I Monitoring network monitoring the actual movements of the lock:

Slide 24

Geodetic applications.. 24 February 20, 2008 The probability of type I and type II errors A geodetic monitoring network of a lock: Measurements: - horizontal directions (a) - zenith angles (b) - distances (c)

Slide 25

Geodetic applications.. 25 February 20, 2008 The probability of type I and type II errors Probability of a type I error in the imprecise case: A single outlier test (weak imprecision):

Slide 26

Geodetic applications.. 26 February 20, 2008 The probability of type I and type II errors A single outlier test (weak imprecision): Probability of a type I error in the imprecise case:

Slide 27

Geodetic applications.. 27 February 20, 2008 The probability of type I and type II errors Probability of a type I error in the imprecise case for a multiple outlier test: Non-centrality parameter for a multiple outlier test:

Slide 28

Geodetic applications.. 28 February 20, 2008 The probability of type I and type II errors epoch 1999epoch 2004 observations317144 parameters6039 6 identical points Congruence Test (strong imprecision):

Slide 29

Geodetic applications.. 29 February 20, 2008 The probability of type I and type II errors Congruence test (strong imprecision): Test situation: Probability of a type I error in the imprecise case for the congruence test:

Slide 30

Statistical hypothesis tests in linear parameter estimation (impecise case) Type I and Type II error probabilities The non-centrality parameter in the imprecise case 1D case is straightforward, mD case needs -cut optimization The difference between the precise and the imprecise case depends on the task of the test: (i) outlier tests or (ii) safety-relevant test and on the order of magnitude of imprecision 30 February 20, 2008 The probability of type I and type II errors Conclusions and future work In progress: Assessment and validation using real data Reduce the computational complexity Take object fuzziness into account

Slide 31

The presented results are developed within the research project KU 1250/4 Geodtische Deformationsanalysen unter Verwendung von Beobachtungsimprzision und Objektun- schrfe, which is funded by the German Research Foundation (DFG). This is gratefully acknowledged Thank you for your attention! 31 February 20, 2008 The probability of type I and type II errors Acknowledgements

Slide 32

Contact information Ingo Neumann and Hansjrg Kutterer Geodetic Institute Leibniz University of Hannover Nienburger Strae 1, D-30167 Hannover, Germany Tel.: +49/+511/762-4394 E-Mail: {neumann, kutterer}@gih.uni-hannover.deneumann, kutterer@gih.uni-hannover.de www.gih.uni-hannover.de The probability of type I and type II errors in imprecise hypothesis testing 32 February 20, 2008 The probability of type I and type II errors

Slide 33

33

Slide 34

Tasks and methods Determination of relevant quantities / parameters Calculation of observation imprecision Propagation of observation imprecision to the est. parameters Assessment of accuracy (imprecise case) Regression and least squares adjustments Statistical hypothesis tests - General form of a linear hypothesis - Probability of Type I/II errors Optimization of configuration 34 February 20, 2008 The probability of type I and type II errors Uncertainty Modeling

Slide 35

Schnitt Optimierung Methoden zur Analyse der Imprzision

Slide 36

cut optimization

Slide 37

1 x Precise case (1D) Example: Two-sided comparison of a mean value with a given value Null hypothesis H 0, alternative hypothesis H A, error probability Definition of regions of acceptance A and rejection R Clear and unique decisions ! 37 February 20, 2008 The probability of type I and type II errors Statistical hypothesis tests

Slide 38

38 February 20, 2008 The probability of type I and type II errors Statistical hypothesis tests 1 x Imprecise case Consideration of imprecision Precise case x 1 Imprecision of test statistics due to the imprecision of the observations

Slide 39

1 xx 1 Fuzzy-interval Imprecise casePrecise case Imprecision of the region of acceptance due to the linguistic fuzziness or modeled regions of transition 39 February 20, 2008 The probability of type I and type II errors Statistical hypothesis tests Consideration of imprecision

Slide 40

40 February 20, 2008 The probability of type I and type II errors Statistical hypothesis tests 11 xx Imprecise casePrecise case Imprecision of the region of rejection as complement of the region of acceptance Consideration of imprecision

Slide 41

11 xx Imprecise casePrecise case Conclusion: Transition regions prevent a clear and unique test decision ! 41 February 20, 2008 The probability of type I and type II errors Statistical hypothesis tests Consideration of imprecision

Slide 42

Conditions for an adequate test strategy Quantitative comparison of the imprecise test statistics and the regions of acceptance and rejection Precise criterion pro or con acceptance Probabilistic interpretation of the results Statistical hypothesis tests