gianluca iaccarino department of mechanical engineering...

Introduction to Uncertainty Quantificationin Computational Science

Handout #1

Gianluca IaccarinoDepartment of Mechanical Engineering

Stanford University

June 29 - July 1, 2009

Scuola di Dottorato di Ricerca in Ingegneria Industriale

Dottorato di Ricerca in Ingegneria Aerospaziale, Navale e della Qualita’

Universita’ di Napoli “Federico II”

Based on the Lecture Notes for ME470 prepared with Dr. A. Doostan

Objectives of the Lectures

I Introduce Uncertainty Quantification (UQ)I Definitions and motivationsI Classification of various techniquesI Application examples

I Characterize mysterious concepts such as:I polynomial chaosI stochastic collocationI surrogate samplingI epistemic uncertaintyI . . .

I Identify the research areas and the connections to otherfields and convey the challenges and the opportunity in UQ

Objectives of the Lectures

I Introduce Uncertainty Quantification (UQ)I Definitions and motivationsI Classification of various techniquesI Application examples

I Characterize mysterious concepts such as:I polynomial chaosI stochastic collocationI surrogate samplingI epistemic uncertaintyI . . .

I Identify the research areas and the connections to otherfields and convey the challenges and the opportunity in UQ

Objectives of the Lectures

I Introduce Uncertainty Quantification (UQ)I Definitions and motivationsI Classification of various techniquesI Application examples

I Characterize mysterious concepts such as:I polynomial chaosI stochastic collocationI surrogate samplingI epistemic uncertaintyI . . .

I Identify the research areas and the connections to otherfields and convey the challenges and the opportunity in UQ

Topics

I Introduction and definitionsI Types of uncertaintiesI Errors vs. uncertaintiesI Sensitivity analysis

I Theoretical frameworkI Probability theoryI Other approaches: Intervals, evidence

I Data representationI Estimate of mean, variance, moments

I Uncertainty propagationI Sampling methods: MC, LHSI Surrogate response: collocationI Polynomial chaos methods

Topics

I Introduction and definitionsI Types of uncertaintiesI Errors vs. uncertaintiesI Sensitivity analysis

I Theoretical frameworkI Probability theoryI Other approaches: Intervals, evidence

I Data representationI Estimate of mean, variance, moments

I Uncertainty propagationI Sampling methods: MC, LHSI Surrogate response: collocationI Polynomial chaos methods

Topics

I Introduction and definitionsI Types of uncertaintiesI Errors vs. uncertaintiesI Sensitivity analysis

I Theoretical frameworkI Probability theoryI Other approaches: Intervals, evidence

I Data representationI Estimate of mean, variance, moments

I Uncertainty propagationI Sampling methods: MC, LHSI Surrogate response: collocationI Polynomial chaos methods

Topics

I Introduction and definitionsI Types of uncertaintiesI Errors vs. uncertaintiesI Sensitivity analysis

I Theoretical frameworkI Probability theoryI Other approaches: Intervals, evidence

I Data representationI Estimate of mean, variance, moments

I Uncertainty propagationI Sampling methods: MC, LHSI Surrogate response: collocationI Polynomial chaos methods

Topics

I Introduction and definitionsI Types of uncertaintiesI Errors vs. uncertaintiesI Sensitivity analysis

I Theoretical frameworkI Probability theoryI Other approaches: Intervals, evidence

I Data representationI Estimate of mean, variance, moments

I Uncertainty propagationI Sampling methods: MC, LHSI Surrogate response: collocationI Polynomial chaos methods

Topics

I Introduction and definitionsI Types of uncertaintiesI Errors vs. uncertaintiesI Sensitivity analysis

I Theoretical frameworkI Probability theoryI Other approaches: Intervals, evidence

I Data representationI Estimate of mean, variance, moments

I Uncertainty propagationI Sampling methods: MC, LHSI Surrogate response: collocationI Polynomial chaos methods

Why Uncertainty Quantification?

Why Uncertainty Quantification?UQ for Decision Making

I In spite of the wide spread use of Modeling and Simulation(M&S) tools it remains difficult to provide objectiveconfidence levels in the quantitative information obtainedfrom numerical predictions

I The complexity arises from the uncertainties related to theinputs of any computation attempting to represent a realphysical system.

I Use of M&S predictions in high-impact decisions require arigorous evaluation of the confidence

Why Uncertainty Quantification?UQ for Decision Making

Example: hurricane forecasting

Image from NOAA

Why Uncertainty Quantification?UQ for Decision Making

Example: Yucca Mountain Nuclear Waste Repository

Image from Peterson, Kastemberg, Corradini, 2004

Observations

I Predictions are strongly subject to uncertaintyI Forecasts are typically presented as probabilities of

outcomes: 20% chance of rainI Whether prediction models are continuously reset:

extensive observation/measurements are used to calibratethe models and to define initial conditions; on the otherhand radiation level models are based on sparse data

I Several competing models of the important physicalprocesses are typically available

Keywords: Data rich/sparse environments, calibration,probabilities, models

Observations

I Predictions are strongly subject to uncertaintyI Forecasts are typically presented as probabilities of

outcomes: 20% chance of rainI Whether prediction models are continuously reset:

extensive observation/measurements are used to calibratethe models and to define initial conditions; on the otherhand radiation level models are based on sparse data

I Several competing models of the important physicalprocesses are typically available

Keywords: Data rich/sparse environments, calibration,probabilities, models

Why Uncertainty Quantification?UQ for Validation

I The accepted process of evaluating M&S tools andsolutions is based on the general concept of Verificationand Validation (V&V)

I The last step of the process is invariably based oncomparisons between numerical predictions and physicalobservations

I Precise quantification of the errors and uncertainties isrequired to establish predictive capabilities: UQ is a keyingredient of validation!

Why Uncertainty Quantification?UQ for Validation

Example: measurements of the speed of light (1870-1960)

Image from Christie et al., Los Alamos Science, #29, 2005

Why Uncertainty Quantification?UQ for Validation

Comparisons of measurements and numerical predictions mustinclude the quantification of the uncertainty

...but also the precise notion of validated model must bediscussed

Images from Trucano et al. 2002, and Romero 2008

Why Uncertainty Quantification?UQ for Validation

Comparisons of measurements and numerical predictions mustinclude the quantification of the uncertainty

...but also the precise notion of validated model must bediscussed

Images from Trucano et al. 2002, and Romero 2008

Why Uncertainty Quantification?UQ for Validation

Comparisons of measurements and numerical predictions mustinclude the quantification of the uncertainty

...but also the precise notion of validated model must bediscussed

Images from Trucano et al. 2002, and Romero 2008

Observations

I Experimentalists are forced to provide a measure of theuncertainty in their data! Computationalists (???) will besoon...

I Comparing two uncertain quantities is not easy? what arethe metrics?

I Only few quantities are measures/computed what can wesay about other quantities?

I Measurements from different experimental campaign canbe inconsistent, contradictory, etc.

I Experiments might only cover a limited parameter space(e.g. Re, Mach, etc.): the validation domain. What if theapplication domain is larger?

Keywords: validation metrics, validation domain, inconsistentdata

Observations

I Experimentalists are forced to provide a measure of theuncertainty in their data! Computationalists (???) will besoon...

I Comparing two uncertain quantities is not easy? what arethe metrics?

I Only few quantities are measures/computed what can wesay about other quantities?

I Measurements from different experimental campaign canbe inconsistent, contradictory, etc.

I Experiments might only cover a limited parameter space(e.g. Re, Mach, etc.): the validation domain. What if theapplication domain is larger?

Keywords: validation metrics, validation domain, inconsistentdata

Why Uncertainty Quantification?UQ for Design/Optimization

I A robust design is one where system performance remainsrelatively unchanged (stable) when exposed touncertainties in the operating conditions.

I Robust design/optimization is a powerful tool for managingthe tradeoffs between optimal performance andperformance stability.

I A reliability-based design is one where the probability offailure is less than some acceptable value

Why Uncertainty Quantification?UQ for Design/Optimization

Example: transonic wing shape optimization

I Choice of design conditions can dramatically affectperformance

I Impact of flight conditions uncertainties can lead tounknown/unexpected consequences

Image from T. Zang, 2003

Why Uncertainty Quantification?One more...

Slide from M. Anderson, LANL

Why Uncertainty Quantification?Simplistic View:Error bars on the numerical results, just like experiments...The results provide an intuitive notion of confidence

Unsteady turbulent heat convection with uncertain wall heating

Constantine & Iaccarino, 2009

One important objective of UQ is to make the intuitive notion ofconfidence mathematically sound!

Why Uncertainty Quantification?Simplistic View:Error bars on the numerical results, just like experiments...The results provide an intuitive notion of confidence

Unsteady turbulent heat convection with uncertain wall heating

Constantine & Iaccarino, 2009

One important objective of UQ is to make the intuitive notion ofconfidence mathematically sound!

Definitions

Verification and ValidationDefinitions

The American Institute for Aeronautics and Astronautics (AIAA)has developed the “Guide for the Verification and Validation(V&V) of Computational Fluid Dynamics Simulations” (1998)

What is V&V?I Verification: The process of determining that a model

implementation accurately represents the developer’sconceptual description of the model.

I Validation The process of determining the degree to whicha model is an accurate representation of the real world forthe intended uses of the model

Errors vs. UncertaintiesDefinitions

The AIAA “Guide for the Verification and Validation (V&V) ofCFD Simulations” (1998) defines

I errors as recognisable deficiencies of the models or thealgorithms employed

I uncertainties as a potential deficiency that is due to lack ofknowledge.

What is the relation with V&V?I Verification aims at answering the question “are we solving

the equations correctly?” – it is an exercise in mathematicsI Validation aims at answering the question “are we solving

the correct equations?” – it is an exercise in physics

Errors vs. UncertaintiesDefinitions

The AIAA “Guide for the Verification and Validation (V&V) ofCFD Simulations” (1998) defines

I errors as recognisable deficiencies of the models or thealgorithms employed

I uncertainties as a potential deficiency that is due to lack ofknowledge.

What is the relation with V&V?

I Verification aims at answering the question “are we solvingthe equations correctly?” – it is an exercise in mathematics

I Validation aims at answering the question “are we solvingthe correct equations?” – it is an exercise in physics

Errors vs. UncertaintiesDefinitions

The AIAA “Guide for the Verification and Validation (V&V) ofCFD Simulations” (1998) defines

I errors as recognisable deficiencies of the models or thealgorithms employed

I uncertainties as a potential deficiency that is due to lack ofknowledge.

What is the relation with V&V?I Verification aims at answering the question “are we solving

the equations correctly?” – it is an exercise in mathematics

I Validation aims at answering the question “are we solvingthe correct equations?” – it is an exercise in physics

Errors vs. UncertaintiesDefinitions

The AIAA “Guide for the Verification and Validation (V&V) ofCFD Simulations” (1998) defines

I errors as recognisable deficiencies of the models or thealgorithms employed

I uncertainties as a potential deficiency that is due to lack ofknowledge.

What is the relation with V&V?I Verification aims at answering the question “are we solving

the equations correctly?” – it is an exercise in mathematicsI Validation aims at answering the question “are we solving

the correct equations?” – it is an exercise in physics

Errors vs. UncertaintiesDefinitions

The AIAA definition is not very precise: it does not clearlydistinguish between the mathematics and the physics.

I What are errors? errors are associated to the translation ofa mathematical formulation into a numerical algorithm anda computational code.

I Acknowledged errors are known to be present but theireffect on the results is deemed negligible (round-off, limitedconvergence of iterative algorithms)

I Unacknowledged errors are not recognizable but might bepresent, e.g. implementation mistakes (bugs).

I What are uncertainties? uncertainties are associated tothe specification of the input physical parameters requiredfor performing the analysis.

I "...an uncertain input parameter will lead not only to anuncertain solution but to an uncertain solution error aswell.." [Trucano, 2004]

Errors vs. UncertaintiesDefinitions

The AIAA definition is not very precise: it does not clearlydistinguish between the mathematics and the physics.

I What are errors? errors are associated to the translation ofa mathematical formulation into a numerical algorithm anda computational code.

I Acknowledged errors are known to be present but theireffect on the results is deemed negligible (round-off, limitedconvergence of iterative algorithms)

I Unacknowledged errors are not recognizable but might bepresent, e.g. implementation mistakes (bugs).

I What are uncertainties? uncertainties are associated tothe specification of the input physical parameters requiredfor performing the analysis.

I "...an uncertain input parameter will lead not only to anuncertain solution but to an uncertain solution error aswell.." [Trucano, 2004]

Errors vs. UncertaintiesDefinitions

The AIAA definition is not very precise: it does not clearlydistinguish between the mathematics and the physics.

I What are errors? errors are associated to the translation ofa mathematical formulation into a numerical algorithm anda computational code.

I Acknowledged errors are known to be present but theireffect on the results is deemed negligible (round-off, limitedconvergence of iterative algorithms)

I Unacknowledged errors are not recognizable but might bepresent, e.g. implementation mistakes (bugs).

I What are uncertainties? uncertainties are associated tothe specification of the input physical parameters requiredfor performing the analysis.

I "...an uncertain input parameter will lead not only to anuncertain solution but to an uncertain solution error aswell.." [Trucano, 2004]

Errors vs. UncertaintiesDefinitions

The AIAA definition is not very precise: it does not clearlydistinguish between the mathematics and the physics.

I What are errors? errors are associated to the translation ofa mathematical formulation into a numerical algorithm anda computational code.

I Acknowledged errors are known to be present but theireffect on the results is deemed negligible (round-off, limitedconvergence of iterative algorithms)

I Unacknowledged errors are not recognizable but might bepresent, e.g. implementation mistakes (bugs).

I What are uncertainties? uncertainties are associated tothe specification of the input physical parameters requiredfor performing the analysis.

I "...an uncertain input parameter will lead not only to anuncertain solution but to an uncertain solution error aswell.." [Trucano, 2004]

Uncertainties

Aleatory: it is the physical variability present in the system or itsenvironment.

I It is not strictly due to a lack of knowledge and cannot bereduced (also referred to as variability, stochastic uncertainty orirreducible uncertainty)

I It is naturally defined in aprobabilistic framework

I Examples are: material properties,operating conditions manufacturingtolerances, etc.

I In mathematical modeling it is alsostudied as noise

Uncertainties

Aleatory: it is the physical variability present in the system or itsenvironment.

I It is not strictly due to a lack of knowledge and cannot bereduced (also referred to as variability, stochastic uncertainty orirreducible uncertainty)

I It is naturally defined in aprobabilistic framework

I Examples are: material properties,operating conditions manufacturingtolerances, etc.

I In mathematical modeling it is alsostudied as noise

Uncertainties

Aleatory: it is the physical variability present in the system or itsenvironment.

I It is not strictly due to a lack of knowledge and cannot bereduced (also referred to as variability, stochastic uncertainty orirreducible uncertainty)

I It is naturally defined in aprobabilistic framework

I Examples are: material properties,operating conditions manufacturingtolerances, etc.

I In mathematical modeling it is alsostudied as noise

Aleatory UncertaintyManufacturing process

Courtesy of NASA

Aleatory UncertaintyFlight conditions

Difference between measured (balloon) and expected (GlobalReference Atmospheric Model) density in the earth atmosphere

Image from Desai et al. 2003

Uncertainties

Epistemic: it is a potential deficiency that is due to a lack ofknowledge

I It can arise from assumptions introduced in the derivation of themathematical model (it is also called reducible uncertainty orincertitude)

I Examples are: turbulencemodel assumptions orsurrogate chemical models

I It is NOT naturally defined ina probabilistic framework

I Can lead to strong bias of thepredictions

Uncertainties

Epistemic: it is a potential deficiency that is due to a lack ofknowledge

I It can arise from assumptions introduced in the derivation of themathematical model (it is also called reducible uncertainty orincertitude)

I Examples are: turbulencemodel assumptions orsurrogate chemical models

I It is NOT naturally defined ina probabilistic framework

I Can lead to strong bias of thepredictions

Uncertainties

Epistemic: it is a potential deficiency that is due to a lack ofknowledge

I It can arise from assumptions introduced in the derivation of themathematical model (it is also called reducible uncertainty orincertitude)

I Examples are: turbulencemodel assumptions orsurrogate chemical models

I It is NOT naturally defined ina probabilistic framework

I Can lead to strong bias of thepredictions

Epistemic UncertaintyModel uncertainty

CBO Predictions of deficit as a percentage of GDP

Source: Congressional Budget Office.

Epistemic UncertaintyModel uncertainty

Predictions of surface pressure over a transonic bump

Reducible vs. Irreducible Uncertainty

I Epistemic uncertainty can be reduced by increasing ourknowledge, e.g. performing more experimentalinvestigations and/or developing new physical models.

I Aleatory uncertainty cannot be reduced as it arisesnaturally from observations of the system. Additionalexperiments can only be used to better characterize thevariability.

Sensitivity Analysis vs. UQ

I Sensitivity analysis (SA) investigates the connectionbetween inputs and outputs of a (computational) model

I The objective of SA is to identify how the variability in anoutput quantity of interest (q) is connected to an input (ξ) inthe model; the result is a sensitivity derivative ∂q/∂ξ

I SA allows to build a ranking of the input sources whichmight dominate the response of the system

I Note that strong (large) sensitivities derivatives do notnecessarily translate in critical uncertainties because theinput variability might be very small in a specific device ofinterest.

∆q ≈ ∂q∂ξ

∆ξ

Sensitivity Analysis vs. UQ

I Sensitivity analysis (SA) investigates the connectionbetween inputs and outputs of a (computational) model

I The objective of SA is to identify how the variability in anoutput quantity of interest (q) is connected to an input (ξ) inthe model; the result is a sensitivity derivative ∂q/∂ξ

I SA allows to build a ranking of the input sources whichmight dominate the response of the system

I Note that strong (large) sensitivities derivatives do notnecessarily translate in critical uncertainties because theinput variability might be very small in a specific device ofinterest.

∆q ≈ ∂q∂ξ

∆ξ

Predictive Scientific Computing

=

Computations Under Uncertainty

Uncertainty QuantificationComputational Framework

Consider a generic computational model (y ∈ <d with d large)

How do we handle the uncertainties?

1. Data assimilation: characterize uncertainties in the inputs

2. Uncertainty propagation: perform simulations accountingfor the identified uncertainties

3. Certification: establish acceptance criteria for predictions

Uncertainty QuantificationComputational Framework

Consider a generic computational model (y ∈ <d with d large)

How do we handle the uncertainties?

1. Data assimilation: characterize uncertainties in the inputs

2. Uncertainty propagation: perform simulations accountingfor the identified uncertainties

3. Certification: establish acceptance criteria for predictions

Uncertainty QuantificationComputational Framework

Consider a generic computational model (y ∈ <d with d large)

How do we handle the uncertainties?

1. Data assimilation: characterize uncertainties in the inputs

2. Uncertainty propagation: perform simulations accountingfor the identified uncertainties

3. Certification: establish acceptance criteria for predictions

Uncertainty QuantificationComputational Framework

Consider a generic computational model (y ∈ <d with d large)

How do we handle the uncertainties?

1. Data assimilation: characterize uncertainties in the inputs

2. Uncertainty propagation: perform simulations accountingfor the identified uncertainties

3. Certification: establish acceptance criteria for predictions

Uncertainty QuantificationComputational Framework

Consider a generic computational model (y ∈ <d with d large)

How do we handle the uncertainties?

1. Data assimilation: characterize uncertainties in the inputs

2. Uncertainty propagation: perform simulations accountingfor the identified uncertainties

3. Certification: establish acceptance criteria for predictions

Data assimilation

The objective is characterize uncertainties in simulation inputs,based on available information

I Data sources- Experimental observations- Theoretical arguments- Expert opinions- etc.

I Mathematical framework for data assimilationI Methods are a function of the amount of data available

- Data-rich environments: e.g. stock market- Sparse data environments: e.g. planetary exploration

I inference: determination of the statistical input parametersthat best represent the available information

Data assimilation

The objective is characterize uncertainties in simulation inputs,based on available information

I Data sources- Experimental observations- Theoretical arguments- Expert opinions- etc.

I Mathematical framework for data assimilationI Methods are a function of the amount of data available

- Data-rich environments: e.g. stock market- Sparse data environments: e.g. planetary exploration

I inference: determination of the statistical input parametersthat best represent the available information

Data assimilation

The objective is characterize uncertainties in simulation inputs,based on available information

I Data sources- Experimental observations- Theoretical arguments- Expert opinions- etc.

I Mathematical framework for data assimilationI Methods are a function of the amount of data available

- Data-rich environments: e.g. stock market- Sparse data environments: e.g. planetary exploration

I inference: determination of the statistical input parametersthat best represent the available information

Data assimilation

The objective is characterize uncertainties in simulation inputs,based on available information

I Data sources- Experimental observations- Theoretical arguments- Expert opinions- etc.

I Mathematical framework for data assimilationI Methods are a function of the amount of data available

- Data-rich environments: e.g. stock market- Sparse data environments: e.g. planetary exploration

I inference: determination of the statistical input parametersthat best represent the available information

Data assimilationWhat is the end result of this phase?

I Identification of all (d) the explicit and hidden parameters(knobs) of the mathematical/computational model: y

I Characterization of the associated level of knowledge

I The mathematical framework for propagating uncertaintiesis dependent on the data representation chosen

I Probabilistic propagationI Interval analysisI Fuzzy logic

Data assimilationWhat is the end result of this phase?

I Identification of all (d) the explicit and hidden parameters(knobs) of the mathematical/computational model: y

I Characterization of the associated level of knowledge

I The mathematical framework for propagating uncertaintiesis dependent on the data representation chosen

I Probabilistic propagationI Interval analysisI Fuzzy logic

Data assimilationWhat is the end result of this phase?

I Identification of all (d) the explicit and hidden parameters(knobs) of the mathematical/computational model: y

I Characterization of the associated level of knowledge

I The mathematical framework for propagating uncertaintiesis dependent on the data representation chosen

I Probabilistic propagationI Interval analysisI Fuzzy logic

Data assimilationWhat is the end result of this phase?

I Identification of all (d) the explicit and hidden parameters(knobs) of the mathematical/computational model: y

I Characterization of the associated level of knowledge

I The mathematical framework for propagating uncertaintiesis dependent on the data representation chosen

I Probabilistic propagationI Interval analysisI Fuzzy logic

Uncertainty Propagation

Perform simulations accounting for the identified uncertainties

I Number of choices:

I Probabilistic vs. Non-probabilistic frameworkI Intrusive vs. Non-Intrusive methodologyI Based on Full model vs. SurrogateI ...

It is difficult to build a proper taxonomy of methods

Uncertainty Propagation

Perform simulations accounting for the identified uncertainties

I Number of choices:I Probabilistic vs. Non-probabilistic frameworkI Intrusive vs. Non-Intrusive methodologyI Based on Full model vs. SurrogateI ...

It is difficult to build a proper taxonomy of methods

Probabilistic Uncertainty Propagation

Perform simulations accounting for the uncertainty representedas randomness

I Define an abstract probability space (Ω,A,P)

I Introduce uncertain input as random quantities y(ω), ω ∈ Ω

I The original problem becomes stochastic with solutionu(ω) ≡ u(y(ω))

Remark: y can affect the boundary conditions, the geometry,the forcing terms or the operator in the computational model.

Probabilistic Uncertainty Propagation

Perform simulations accounting for the uncertainty representedas randomness

I Define an abstract probability space (Ω,A,P)

I Introduce uncertain input as random quantities y(ω), ω ∈ Ω

I The original problem becomes stochastic with solutionu(ω) ≡ u(y(ω))

Remark: y can affect the boundary conditions, the geometry,the forcing terms or the operator in the computational model.

Probabilistic Uncertainty Propagation

Perform simulations accounting for the uncertainty representedas randomness

I Define an abstract probability space (Ω,A,P)

I Introduce uncertain input as random quantities y(ω), ω ∈ Ω

I The original problem becomes stochastic with solutionu(ω) ≡ u(y(ω))

Remark: y can affect the boundary conditions, the geometry,the forcing terms or the operator in the computational model.

Probabilistic Uncertainty Propagation

Perform simulations accounting for the uncertainty representedas randomness

I Define an abstract probability space (Ω,A,P)

I Introduce uncertain input as random quantities y(ω), ω ∈ Ω

I The original problem becomes stochastic with solutionu(ω) ≡ u(y(ω))

Remark: y can affect the boundary conditions, the geometry,the forcing terms or the operator in the computational model.

Uncertainty PropagationIntrusive vs. Non-Intrusive Methodology

Intrusiveness is defined with respect to the original(deterministic) mathematical model, not the physicaldescription of the problem

I Nonintrusive methods only require (multiple) solutions ofthe original model

I Intrusive methods require the formulation and solution of astochastic version of the original model

Uncertainty PropagationIntrusive vs. Non-Intrusive Methodology

Intrusiveness is defined with respect to the original(deterministic) mathematical model, not the physicaldescription of the problem

I Nonintrusive methods only require (multiple) solutions ofthe original model

I Intrusive methods require the formulation and solution of astochastic version of the original model

Uncertainty PropagationIntrusive vs. Non-Intrusive Methodology

Intrusiveness is defined with respect to the original(deterministic) mathematical model, not the physicaldescription of the problem

I Nonintrusive methods only require (multiple) solutions ofthe original model

I Intrusive methods require the formulation and solution of astochastic version of the original model

Example of intrusive methodsPerturbation methods

Consider a very simple mathematical model to study 1Dnon-linear flow motion (Burgers’ equation):

∂u∂t

+ u∂u∂x

= 0 x ∈ [0 : 1]

with uncertain initial conditions u(x , t = 0) = u0(ω)

I We seek the solution u(x , t , ω)

I Define a base state 〈u(x , t)〉 and write u(x , t , ω) ≡ 〈u〉+ δuI Plug in the original equation to obtain:

∂δu∂t

+ 〈u〉∂δu∂x

+ δu∂〈u〉∂x

= 0(

given∂〈u〉∂t

+ 〈u〉∂〈u〉∂x

= 0)

I The key assumption is δu2 ≈ 0I The method implies the original problem for the base state

and a new, linear problem for the perturbation

Example of intrusive methodsPerturbation methods

Consider a very simple mathematical model to study 1Dnon-linear flow motion (Burgers’ equation):

∂u∂t

+ u∂u∂x

= 0 x ∈ [0 : 1]

with uncertain initial conditions u(x , t = 0) = u0(ω)

I We seek the solution u(x , t , ω)

I Define a base state 〈u(x , t)〉 and write u(x , t , ω) ≡ 〈u〉+ δuI Plug in the original equation to obtain:

∂δu∂t

+ 〈u〉∂δu∂x

+ δu∂〈u〉∂x

= 0(

given∂〈u〉∂t

+ 〈u〉∂〈u〉∂x

= 0)

I The key assumption is δu2 ≈ 0I The method implies the original problem for the base state

and a new, linear problem for the perturbation

Example of intrusive methodsPerturbation methods

Consider a very simple mathematical model to study 1Dnon-linear flow motion (Burgers’ equation):

∂u∂t

+ u∂u∂x

= 0 x ∈ [0 : 1]

with uncertain initial conditions u(x , t = 0) = u0(ω)

I We seek the solution u(x , t , ω)

I Define a base state 〈u(x , t)〉 and write u(x , t , ω) ≡ 〈u〉+ δu

I Plug in the original equation to obtain:

∂δu∂t

+ 〈u〉∂δu∂x

+ δu∂〈u〉∂x

= 0(

given∂〈u〉∂t

+ 〈u〉∂〈u〉∂x

= 0)

I The key assumption is δu2 ≈ 0I The method implies the original problem for the base state

and a new, linear problem for the perturbation

Example of intrusive methodsPerturbation methods

Consider a very simple mathematical model to study 1Dnon-linear flow motion (Burgers’ equation):

∂u∂t

+ u∂u∂x

= 0 x ∈ [0 : 1]

with uncertain initial conditions u(x , t = 0) = u0(ω)

I We seek the solution u(x , t , ω)

I Define a base state 〈u(x , t)〉 and write u(x , t , ω) ≡ 〈u〉+ δuI Plug in the original equation to obtain:

∂δu∂t

+ 〈u〉∂δu∂x

+ δu∂〈u〉∂x

= 0(

given∂〈u〉∂t

+ 〈u〉∂〈u〉∂x

= 0)

I The key assumption is δu2 ≈ 0I The method implies the original problem for the base state

and a new, linear problem for the perturbation

Example of intrusive methodsPerturbation methods

Consider a very simple mathematical model to study 1Dnon-linear flow motion (Burgers’ equation):

∂u∂t

+ u∂u∂x

= 0 x ∈ [0 : 1]

with uncertain initial conditions u(x , t = 0) = u0(ω)

I We seek the solution u(x , t , ω)

I Define a base state 〈u(x , t)〉 and write u(x , t , ω) ≡ 〈u〉+ δuI Plug in the original equation to obtain:

∂δu∂t

+ 〈u〉∂δu∂x

+ δu∂〈u〉∂x

= 0(

given∂〈u〉∂t

+ 〈u〉∂〈u〉∂x

= 0)

I The key assumption is δu2 ≈ 0

I The method implies the original problem for the base stateand a new, linear problem for the perturbation

Example of intrusive methodsPerturbation methods

Consider a very simple mathematical model to study 1Dnon-linear flow motion (Burgers’ equation):

∂u∂t

+ u∂u∂x

= 0 x ∈ [0 : 1]

with uncertain initial conditions u(x , t = 0) = u0(ω)

I We seek the solution u(x , t , ω)

I Define a base state 〈u(x , t)〉 and write u(x , t , ω) ≡ 〈u〉+ δuI Plug in the original equation to obtain:

∂δu∂t

+ 〈u〉∂δu∂x

+ δu∂〈u〉∂x

= 0(

given∂〈u〉∂t

+ 〈u〉∂〈u〉∂x

= 0)

I The key assumption is δu2 ≈ 0I The method implies the original problem for the base state

and a new, linear problem for the perturbation

Why intrusive methods?

I Perturbation methods are clearly strongly limited, but theassumption of small variability can be lifted

I Intrusive methods can be strongly customized, highlyeffective, and provide a connection to the physics/modelingof the problem

I For realistic engineering problems intrusive methodsremain challenging to implement, expensive to use anddifficult to verify

I Answering the question "to intrude or not to intrude"remains a key research area and likely a case-by-casechoice.

Why intrusive methods?

I Perturbation methods are clearly strongly limited, but theassumption of small variability can be lifted

I Intrusive methods can be strongly customized, highlyeffective, and provide a connection to the physics/modelingof the problem

I For realistic engineering problems intrusive methodsremain challenging to implement, expensive to use anddifficult to verify

I Answering the question "to intrude or not to intrude"remains a key research area and likely a case-by-casechoice.

Why intrusive methods?

I Perturbation methods are clearly strongly limited, but theassumption of small variability can be lifted

I Intrusive methods can be strongly customized, highlyeffective, and provide a connection to the physics/modelingof the problem

I For realistic engineering problems intrusive methodsremain challenging to implement, expensive to use anddifficult to verify

I Answering the question "to intrude or not to intrude"remains a key research area and likely a case-by-casechoice.

Uncertainty PropagationFull model vs. Surrogate

Surrogates or Reduced Order Models (ROM) are based on asimplification of the original mathematical problem

I Examples of ROMs:I Coarse gridsI Subset of the physical phenomenaI Curve fit of the output quantity of interest (response

surface)I . . .

I Remark: the use of ROMs is only justified by the cost ofUQ for the full system.

Uncertainty PropagationFull model vs. Surrogate

Surrogates or Reduced Order Models (ROM) are based on asimplification of the original mathematical problem

I Examples of ROMs:I Coarse gridsI Subset of the physical phenomenaI Curve fit of the output quantity of interest (response

surface)I . . .

I Remark: the use of ROMs is only justified by the cost ofUQ for the full system.

Uncertainty PropagationFull model vs. Surrogate

Surrogates or Reduced Order Models (ROM) are based on asimplification of the original mathematical problem

I Examples of ROMs:I Coarse gridsI Subset of the physical phenomenaI Curve fit of the output quantity of interest (response

surface)I . . .

I Remark: the use of ROMs is only justified by the cost ofUQ for the full system.

CertificationThe last step in UQ is also the most important:

I Quantification of the confidence for decision makingI Objective analysis of the validity of a

physical/mathematical modelI . . .

How to report the wealth ofinformation gathered through a UQprocess? For example in aprobabilistic framework we can useboxplots

In a decision-making context we need to provide a simple andclear summary.

CertificationThe last step in UQ is also the most important:

I Quantification of the confidence for decision makingI Objective analysis of the validity of a

physical/mathematical modelI . . .

How to report the wealth ofinformation gathered through a UQprocess? For example in aprobabilistic framework we can useboxplots

In a decision-making context we need to provide a simple andclear summary.

CertificationThe last step in UQ is also the most important:

I Quantification of the confidence for decision makingI Objective analysis of the validity of a

physical/mathematical modelI . . .

How to report the wealth ofinformation gathered through a UQprocess? For example in aprobabilistic framework we can useboxplots

In a decision-making context we need to provide a simple andclear summary.

Quantification of Margins and Uncertainty: QMU

The concept of QMU has been introduced to facilitatecommunication of numerical results:

I for a quantity of interest (q)define a performance gate(acceptable operation)

I predict q for the designconditions (best estimate)

I evaluate the marginI perform an uncertainty analysisI define the Confidence ≈ M/U

From Sharp & Wood-Shultz, 2003

Remark: in general both the predictions and the bounds areuncertain.

Quantification of Margins and Uncertainty: QMU

The concept of QMU has been introduced to facilitatecommunication of numerical results:

I for a quantity of interest (q)define a performance gate(acceptable operation)

I predict q for the designconditions (best estimate)

I evaluate the marginI perform an uncertainty analysisI define the Confidence ≈ M/U

From Sharp & Wood-Shultz, 2003

Remark: in general both the predictions and the bounds areuncertain.

Example:design a vehicle to explore Titan

Planet exploration vehicle

I The exploration of Saturn’s moonTitan was one of the objective ofthe Huygens mission by NASAand ESA

I During descent in theatmosphere vehicles experienceextreme heating loadsI The design of the thermal protection system (TPS) is the

most critical component of every planetary entry missionI TPS design is fundamentally computation-based because

no ground-test can reproduce all the aspects of flightI Safety (and reliability) requires rigorous evaluation of the

uncertainties

Planetary Exploration Vehicles

Predictions of TPS heating loads during re-entry arechallenging

I Physics Components- Chemistry- Radiation- Turbulence- etc.

I Computational issues- Strong shocks- Thin boundary layers- Flow separation- etc.

In addition, CFD simulations require precise input parameters,but the real vehicle is subject to uncertain conditions...

Uncertainty sources (aleatory)Predicting heating rates

I Imprecise characterization of the environment- speed of sound- angle of attack- temperature fluctuations- gas composition- etc.

I Static characterization of the vehicle components- structure and material imperfections/inhomogeneity- surface rougheness and out-of-spec geometry- etc.

I Thermal-fluid processes during operation- ablation rates- surface catalysis- etc.

Uncertainty sources (epistemic)Predicting heating rates

I Physical modeling- High temperature gas properties - dissociation, ionization- Radiation- Transition to turbulence- Surface chemistry- etc.

I Missing physics (unknown unknowns)- Non-continuum effects- Thermal distortion of the structure- Charring of the ablating material- etc.

Probabilistic approachReaction rates in combustion models

I Chemical kinetics: 100s ofspecies and 1000s ofreactions

I Uncertainty in the reactionsrates, gathered from

I theoryI experimentsI engineering judgment

I Uncertainty in the reactionrates is described usingGaussian independentvariables

P(kr ) = exp

[−1

2

(log10kr/E [kr ]

σr

)2]

From Bose & Wright, 2004

Uncertainty analysisThe analysis is carried out using Monte Carlo sampling(≈ 300r .v .) and ≈ 4000 simulations (samples) are required toachieve statistical convergence.

From Bose & Wright, 2004

Primary Uncertainty

NASA [Bose & Wright, 2004] has identified the heating fromshock layer radiation due to the CN radical formed in theN2/CH4 atmosphere as the primary uncertainty

Can we predict the CN radical?

I State-of-the art knowledge duringthe design of the Huygen’s probewas the Boltzmann model

I This led to overprediction of theheating rates - conservativedesign

I Stimulated work oncollisional-radiative (CR) modelsand gathering of newexperimental data From Magin et al., 2007

Primary Uncertainty

NASA [Bose & Wright, 2004] has identified the heating fromshock layer radiation due to the CN radical formed in theN2/CH4 atmosphere as the primary uncertaintyCan we predict the CN radical?

I State-of-the art knowledge duringthe design of the Huygen’s probewas the Boltzmann model

I This led to overprediction of theheating rates - conservativedesign

I Stimulated work oncollisional-radiative (CR) modelsand gathering of newexperimental data From Magin et al., 2007

Primary Uncertainty

NASA [Bose & Wright, 2004] has identified the heating fromshock layer radiation due to the CN radical formed in theN2/CH4 atmosphere as the primary uncertaintyCan we predict the CN radical?

I State-of-the art knowledge duringthe design of the Huygen’s probewas the Boltzmann model

I This led to overprediction of theheating rates - conservativedesign

I Stimulated work oncollisional-radiative (CR) modelsand gathering of newexperimental data From Magin et al., 2007

Primary Uncertainty

We repeated the evaluation of the TPS heating loads using theCR and the Boltzmann model for radiation in an effort tocharacterize the epistemic uncertainty

To first order the results are consistent

Maximum heat flux

CR Model: Ghaffari, Iaccarino, Magin, 2009 Boltzmann model: Bose & Wright, 2004

Primary Uncertainty

We repeated the evaluation of the TPS heating loads using theCR and the Boltzmann model for radiation in an effort tocharacterize the epistemic uncertaintyTo first order the results are consistent

Maximum heat flux

CR Model: Ghaffari, Iaccarino, Magin, 2009 Boltzmann model: Bose & Wright, 2004

Primary Uncertainty

Ranking on the uncertainty sources is based on correlationplots of output (amount of CN) vs. input (uncertainty in thereaction rates)

Correlation plot for N2 + C → CN + N2

CR Model: Ghaffari, Iaccarino, Magin, 2009 Boltzmann model: Bose & Wright, 2004

Primary Uncertainty

Ranking on the uncertainty sources is based on correlationplots of output (amount of CN) vs. input (uncertainty in thereaction rates)

Correlation plot for N2 + C → CN + N2

CR Model: Ghaffari, Iaccarino, Magin, 2009 Boltzmann model: Bose & Wright, 2004

Primary Uncertainty

The decision-making might be affected:Ranking on the uncertainty sources is potentially changed bythe new physical model

8 major contributors to uncertainty

CR Model: Ghaffari, Iaccarino, Magin, 2009 Boltzmann model: Bose & Wright, 2004

Planetary Exploration VehiclesUncertainty analysis

Take home messages:I Quantification of the effect of input uncertainties can

provide useful information and ranking of the sources

thisenables focused research efforts in the areas of high return

I Epistemic uncertainty (physical modeling) can be adominant source of uncertainty and can even affect theranking

I For this problem, a prediction of the stagnation point heatflux can be carried out very efficiently and this enables theuse of Monte Carlo (1000s repeated computations)

Planetary Exploration VehiclesUncertainty analysis

Take home messages:I Quantification of the effect of input uncertainties can

provide useful information and ranking of the sources thisenables focused research efforts in the areas of high return

I Epistemic uncertainty (physical modeling) can be adominant source of uncertainty and can even affect theranking

I For this problem, a prediction of the stagnation point heatflux can be carried out very efficiently and this enables theuse of Monte Carlo (1000s repeated computations)

Planetary Exploration VehiclesUncertainty analysis

Take home messages:I Quantification of the effect of input uncertainties can

provide useful information and ranking of the sources thisenables focused research efforts in the areas of high return

I Epistemic uncertainty (physical modeling) can be adominant source of uncertainty and can even affect theranking

I For this problem, a prediction of the stagnation point heatflux can be carried out very efficiently and this enables theuse of Monte Carlo (1000s repeated computations)

Planetary Exploration VehiclesUncertainty analysis

Take home messages:I Quantification of the effect of input uncertainties can

provide useful information and ranking of the sources thisenables focused research efforts in the areas of high return

I Epistemic uncertainty (physical modeling) can be adominant source of uncertainty and can even affect theranking

I For this problem, a prediction of the stagnation point heatflux can be carried out very efficiently and this enables theuse of Monte Carlo (1000s repeated computations)

Example:analysis of engine intakes

in a Formula 1 car

Fomula 1 Vehicles

I The aerodynamics ofFomula 1 cars is verysophisticated and highlyoptimized

I One of the few aspects thatcannot be actively modifiedis the wake of the tires

I The air intakes are critical to achieve engine performanceI They might be critically affected by the flow in the tire

wakes

Formula 1 Vehicles

Predictions of the air flow in the engine intakes is challengingI Physics Components

- Unsteadiness- Complex geometry- Turbulence- etc.

I Computational issues- Rotating tires and flow in brake ducts- Tire contact patch- Flow separation- etc.

As for the previous example, CFD simulations require preciseinput parameters, but the real vehicle is subject to uncertainconditions...

Uncertainty sourcesIntake air flow

I Imprecise characterizationof the environment

- Incoming disturbancesfrom preceding cars

- Tire deformation- Speed- etc.

I Physical models- turbulence models- multiple rotating frame of

references- etc.

Uncertainty sourcesIntake air flow

The geometry of the tire is uncertainI Aleatory or epistemic?

I We could reduce it by measuring it?? or predict thedeformation using a model! In this case it is likely that wemight introduce more uncertainties, e.g. elastic property ofthe rubber, tire/road slippage, etc.

I How many degrees of freedom do we need to include todescribe the generic shape?

Uncertainty sourcesIntake air flow

The geometry of the tire is uncertainI Aleatory or epistemic?I We could reduce it by measuring it??

or predict thedeformation using a model! In this case it is likely that wemight introduce more uncertainties, e.g. elastic property ofthe rubber, tire/road slippage, etc.

I How many degrees of freedom do we need to include todescribe the generic shape?

Uncertainty sourcesIntake air flow

The geometry of the tire is uncertainI Aleatory or epistemic?I We could reduce it by measuring it?? or predict the

deformation using a model!

In this case it is likely that wemight introduce more uncertainties, e.g. elastic property ofthe rubber, tire/road slippage, etc.

I How many degrees of freedom do we need to include todescribe the generic shape?

Uncertainty sourcesIntake air flow

The geometry of the tire is uncertainI Aleatory or epistemic?I We could reduce it by measuring it?? or predict the

deformation using a model! In this case it is likely that wemight introduce more uncertainties, e.g. elastic property ofthe rubber, tire/road slippage, etc.

I How many degrees of freedom do we need to include todescribe the generic shape?

Uncertainty sourcesIntake air flow

The geometry of the tire is uncertainI Aleatory or epistemic?I We could reduce it by measuring it?? or predict the

deformation using a model! In this case it is likely that wemight introduce more uncertainties, e.g. elastic property ofthe rubber, tire/road slippage, etc.

I How many degrees of freedom do we need to include todescribe the generic shape?

Uncertainty sourcesUncertainty representation

The geometry of the tire isuncertain!

I We choose to represent itusing few (3) parameters

I The justification was asingle measurement of thecontact patch for astationary tire

Experimental "measure"

Uncertainty sourcesUncertainty representation

The geometry of the tire isuncertain!

I We choose to represent itusing few (3) parameters

I The justification was asingle measurement of thecontact patch for astationary tire

Experimental "measure"

Uncertainty sourcesUncertainty representation

The geometry of the tire isuncertain!

I We choose to represent itusing few (3) parameters

I The justification was asingle measurement of thecontact patch for astationary tire

Experimental "measure"

Engine Intake AnalysisUncertainty propagation

I We used a responsesurface approach basedon stochastic collocation

I We can now estimate thestandard deviation of theentire flow field withrespect to the assumedvariability in the input, e.g.the three parametersrepresenting the tiredeformation

Engine Intake AnalysisUncertainty propagation

I We used a responsesurface approach basedon stochastic collocation

I We can now estimate thestandard deviation of theentire flow field withrespect to the assumedvariability in the input, e.g.the three parametersrepresenting the tiredeformation

Engine Intake AnalysisUncertainty propagation

I We can alsoestimate theprobability of the airintake to provide acertain amount offlow rate to theengine

I We can evaluate theprobability of failureassociate to theuncertainty in thetire deformation

Engine Intake AnalysisUncertainty propagation

I We can alsoestimate theprobability of the airintake to provide acertain amount offlow rate to theengine

I We can evaluate theprobability of failureassociate to theuncertainty in thetire deformation

Formula 1 VehiclesUncertainty analysis

Take home messages:I Quantification of the effect of input uncertainties can lead

to different aerodynamic choices

I The representation of the uncertainties is an importantstep: an increase in the number of degrees of freedomconsidered can lead to exponential growth of thecomputational effort required

I For typical CFD problems (3D, complex geometry, etc) thecost is prohibitive: 1000s repeated computations are NOTfeasible

I Changes in the geometry are also associated to changesin the grid resolution and quality. Are we introducinguncertain numerical errors?

Formula 1 VehiclesUncertainty analysis

Take home messages:I Quantification of the effect of input uncertainties can lead

to different aerodynamic choicesI The representation of the uncertainties is an important

step: an increase in the number of degrees of freedomconsidered can lead to exponential growth of thecomputational effort required

I For typical CFD problems (3D, complex geometry, etc) thecost is prohibitive: 1000s repeated computations are NOTfeasible

I Changes in the geometry are also associated to changesin the grid resolution and quality. Are we introducinguncertain numerical errors?

Formula 1 VehiclesUncertainty analysis

Take home messages:I Quantification of the effect of input uncertainties can lead

to different aerodynamic choicesI The representation of the uncertainties is an important

step: an increase in the number of degrees of freedomconsidered can lead to exponential growth of thecomputational effort required

I For typical CFD problems (3D, complex geometry, etc) thecost is prohibitive: 1000s repeated computations are NOTfeasible

I Changes in the geometry are also associated to changesin the grid resolution and quality. Are we introducinguncertain numerical errors?

Formula 1 VehiclesUncertainty analysis

Take home messages:I Quantification of the effect of input uncertainties can lead

to different aerodynamic choicesI The representation of the uncertainties is an important

step: an increase in the number of degrees of freedomconsidered can lead to exponential growth of thecomputational effort required

I For typical CFD problems (3D, complex geometry, etc) thecost is prohibitive: 1000s repeated computations are NOTfeasible

I Changes in the geometry are also associated to changesin the grid resolution and quality. Are we introducinguncertain numerical errors?

Example:estimate flight conditions

during HyShot flight

Validation against flight data

I Flight data are necessarilyone-of-a-kind!

I Only some quantities can be measured,typically do not provide conclusiveevidence.

I Hyshot is an experimental vehicles to study hypersonicflight and to correlate flight-dat and ground tests

I The launch occurred in 2002 in Queensland

HyShot Vehicle

Predictions of the reacting flow in hypersonic vehicles isextremely challenging

I Physics Components- Hypersonic speed, shocks, high-temperature behavior of

the air- Fuel injection, mixing and reactions- Turbulence- etc.

I Computational issues- Strong shocks- Combustion- Flow separation- etc.

To validate our predictions we need to gather the flight data andreproduce the same flow conditions

HyShot VehicleFlight tests

During flightI pressure and

temperaturemeasurement in thecombustionchamber

I condition in the fueltank allow to buildthe fuee flow rate asfunction of time

Unfortunately, the radar telemetry did not work and theinformation about the altitude, speed, etc. were lost

HyShot VehicleFlight tests

During flightI pressure and

temperaturemeasurement in thecombustionchamber

I condition in the fueltank allow to buildthe fuee flow rate asfunction of time

Unfortunately, the radar telemetry did not work and theinformation about the altitude, speed, etc. were lost

HyShot VehicleFlight tests

Problem: given the noisy measurements within the system,can we infer the flight conditions?

Solution: pose a statistical inverse problem, where thesolution is the most probable flight conditionsRemark: need a model that predicts the flow in the systemgiven the flight condition (forward model).

Consider a simplified configuration

We have the exact inflow but want to recover them usingstatistical inversion, we also assume that the measurementsare noisy...

HyShot VehicleFlight tests

Problem: given the noisy measurements within the system,can we infer the flight conditions?Solution: pose a statistical inverse problem, where thesolution is the most probable flight conditions

Remark: need a model that predicts the flow in the systemgiven the flight condition (forward model).

Consider a simplified configuration

We have the exact inflow but want to recover them usingstatistical inversion, we also assume that the measurementsare noisy...

HyShot VehicleFlight tests

Problem: given the noisy measurements within the system,can we infer the flight conditions?Solution: pose a statistical inverse problem, where thesolution is the most probable flight conditionsRemark: need a model that predicts the flow in the systemgiven the flight condition (forward model).

Consider a simplified configuration

We have the exact inflow but want to recover them usingstatistical inversion, we also assume that the measurementsare noisy...

HyShot VehicleFlight tests

Problem: given the noisy measurements within the system,can we infer the flight conditions?Solution: pose a statistical inverse problem, where thesolution is the most probable flight conditionsRemark: need a model that predicts the flow in the systemgiven the flight condition (forward model).

Consider a simplified configuration

We have the exact inflow but want to recover them usingstatistical inversion, we also assume that the measurementsare noisy...

HyShot VehicleSimplified configuration

Estimate of the inflow Mach number and static pressure using 2noisy pressure measurements

HyShot VehicleSimplified configuration

Estimate of the inflow Mach number and static pressure using 4noisy pressure measurements

HyShot Flight DataValidation analysis

Take home messages:I Incomplete information from experimental analysis are

common, but this might not be a show-stopper

I The inverse analysis is based on a forward model, so thefinal estimates are conditioned (dependent) on the modelchosen (and its potential inaccuracy)

I If all the experimental data is used to infer the operatingconditions, nothing is left for validation

HyShot Flight DataValidation analysis

Take home messages:I Incomplete information from experimental analysis are

common, but this might not be a show-stopperI The inverse analysis is based on a forward model, so the

final estimates are conditioned (dependent) on the modelchosen (and its potential inaccuracy)

I If all the experimental data is used to infer the operatingconditions, nothing is left for validation

HyShot Flight DataValidation analysis

Take home messages:I Incomplete information from experimental analysis are

common, but this might not be a show-stopperI The inverse analysis is based on a forward model, so the

final estimates are conditioned (dependent) on the modelchosen (and its potential inaccuracy)

I If all the experimental data is used to infer the operatingconditions, nothing is left for validation

Further Reading Material

1. T. G. Trucano, Prediction and Uncertainty in ComputationalModeling of Complex Phenomena: A Whitepaper, SANDIAReport, SAND98-2776

2. T.G. Trucano, L. P. Swiler, T. Igusa, W. L. Oberkampf, M.Pilch, Calibration, Validation and Sensitivity Analysis:What’s What, Reliability Engineering and System Safety, V.91, 2006

Basic Notions

of Probability Theory

The different kinds of probability

There are four kinds of probability.

A. Probability as intuition

Deals with judgements based on intuition. Examples are:I “She will probably marry him.”I “He probably drove too fast.”I People buying lottery tickets intuitively believe that certain

numbers are more likely to win.

The different kinds of probability

B. Probability as ratio of favorable to total outcomes(classical theory)

This is a non-experimental approach where probability of anevent is computed a priori by counting the number of ways NAthat an event A can occur among N number of all possibleoutcomes. Then the probability of event A is

P(A) ≡ NA

N.

It is considered that all outcomes are equally probable.

The different kinds of probabilityExample: The die rolling

Suppose we throw a pair of fair dice. What is the probability ofhaving a seven?

Outcome of throwing two dice

P(getting seven) =636

=16.

The different kinds of probability

C. Probability as a measure of frequency of occurrence

This is an experimental approach where an experiment isperformed n times. The number of times that event A appearsis denoted by nA. Then the probability of A is postulated as

P(A) = limn→∞

nA

n.

Notice that:

I 0 ≤ P(A) ≤ 1I We can only estimate P(A) from finite number of trials.

Example: One could experimentally estimate the probability ofgetting seven in the die rolling example.

The different kinds of probability

D. Probability based on an axiomatic theory

In this approach probability is defined based on set, field andevent concepts. The probability assignments have to satisfysome constraints called axioms.

I Modern probability theory is based on this approach

Sets, Subsets, and Events

Some definitions:I Set is a collection of objects.I Subset of a set is a collection that is contained within the

larger set.I Sample space/Set of elementary events/Certain event is

the set of all outcomes of an experiment and is denoted byΩ.

I Example: The sample space of the experiment of flipping acoin once is Ω = H,T

I Example: The sample space of the experiment of flipping acoin twice is Ω = HH,HT ,TH,TT.

I Subsets of Ω are called events.I Example: A subset of Ω is HH,HT ,TH which is the event

of getting at least one head in two flips.I Remark: If Ω = ω1, · · · , ωN, then the number of subsets

of Ω is 2N .

Set Algebra

I Union/Sum of two sets A and B is the set of all elementsthat are in at least A or B and is denoted by A ∪ B.

I Intersection of two sets A and B is the set of all elementsthat are common to both A and B and is denoted by A ∩ B.

I Complement of a set A is the set of all elements not in Aand is denoted by Ac . Notice that,

A ∪ Ac = Ω and A ∩ Ac = φ,

where φ is called the impossible event or the empty set.I · · ·

Fields and Sigma Fields

Consider a set Ω and a collection of subsets of Ω. Let A,B, . . .denote subsets in this collection. This collection of subsetsforms a fieldM if

1. φ ∈M, Ω ∈M2. If A ∈M and B ∈M, then A ∪ B ∈M and A ∩ B ∈M3. If A ∈M, then Ac ∈M

A sigma (σ)-field F is a field that is closed under any countablyinfinite set of unions and intersections. Thus if A1, · · · ,An, · · ·belong to F so do

∪∞i=1Ai and ∩∞i=1 Ai .

Probability Space/Probability Triple

Definition:A probability space is a triple (Ω,F ,P) consisting of the samplespace Ω, a σ-field F of subsets of Ω, and a probability functionP on (Ω,F). The probability function P : F → [0,1] is definedbased on the following three axioms.

Axioms of probability:For every event A ∈ F , a number P(A) called the probability ofA is assigned such that:

I P(A) > 0I P(Ω) = 1I P(A ∪ B) = P(A) + P(B) if A ∩ B = φ

Example - Tossing a Coin

Consider the experiment of throwing a coin once. The samplespace Ω is

Ω = H,T.

The σ-field of events consists of sets:

F = H, T,Ω, φ .

For a fair coin, probabilities of events in F are:

P(H) = P(T) = 1/2,

P(Ω) = 1 and P(φ) = 0.

Conditional Probabilities and Independence

Probability of an event A given that an event B has happened(conditional probability) is denoted by P(A|B) and is computedby

P(A|B) =P(A ∩ B)

P(B), P(B) > 0.

Event A is said to be independent from event B if:

P(A|B) = P(A) or P(A ∩ B) = P(A)P(B).

Bayes’ rule:

P(A|B) =P(B|A)P(A)

P(B)

Random Variables

A random variable X (ω) : Ω→ R is a function that maps anoutcome ω ∈ Ω of the random experiment to a number on thereal line. Under such mapping an event AB ⊂ Ω (AB ∈ F) ismapped to an interval B on the real line.

Probability Distribution Function(PDF/CDF)

FX (x) ≡ P(X ≤ x) ≡ P(ω : X (ω) ≤ x)

is called the probability distribution function (PDF) of randomvariable X and has the following properties:

I FX (∞) = 1,FX (−∞) = 0I FX (x) is a nondecreasing function of x, i.e.,

x1 ≤ x2 → FX (x1) ≤ FX (x2)

I FX (x) is continuous from right, i.e.,

FX (x) = limε→0

FX (x + ε) ε > 0

Probability Density Function(pdf)

If FX (x) is continuous and differentiable, the probability densityfunction (pdf) is computed as

fX (x) ≡ dFX (x)

dx,

and has the following properties:

I fX (x) ≥ 0I∫∞−∞ fX (ξ)dξ = FX (∞)− FX (−∞) = 1

I FX (x) =∫ x−∞ fX (ξ)dξ = P(X ≤ x)

I FX (x2)− FX (x1) =∫ x2−∞ fX (ξ)dξ −

∫ x1−∞ fX (ξ)dξ =∫ x2

x1fX (ξ)dξ = P(x1 < X ≤ x2)

Continuous Random Variables

If FX (x) is continuous, then X is said to be a continuousrandom variable.

Examples:I Gaussian random variableI Uniform random variable

Gaussian (Normal) and Uniform Random Variables

Gaussian: X ∼ N(µ, σ2)

fX (x) =1√2πσ

e−12 [ x−µ

σ]2

Standard normal⇒ X ∼ N(0, 1)

Uniform: X ∼ U(a,b)

fX (x) =1

b − aI[a,b](x)

Mean, Variance, and Moments

Mean/Expectation:〈X 〉 ≡ E(X ) ≡ µ(X ) ≡ X ≡

∫ ∞−∞

xfX (x)dx

Variance:

Var(X ) ≡ σ2(X ) ≡⟨(X − 〈X 〉)2⟩ = 〈X 2〉 − 〈X 〉2

≡∫ ∞−∞

x2fX (x)dx −(∫ ∞−∞

xfX (x)dx)2

K-th moment:mk (X ) ≡ 〈X k 〉 =

∫ ∞−∞

xk fX (x)dx , k = 1,2,3, · · ·

Functions of a Random Variable

Let X be a random variable and g(x) a differentiable function ofx , then Y = g(X ) is a random variable with

FY (y) = P(Y ≤ y) = P(g(X ) ≤ y).

The pdf of Y is computed from

fY (y) =n∑

i=1

fX (xi)/|g′(xi)| xi = xi(y), g

′(xi) 6= 0,

where xi are roots of y = g(x) and g′(xi) ≡ dy/dx |x=xi .

Remark:

〈Y = g(X )〉 =

∫DY

yfY (y)dy =

∫DX

g(x)fX (x)dX ,

where DY and DX are the domains of Y and X , respectively.

Two Random Variables

Let X and Y be two random variables. The joint distribution ofX and Y is defined by

FXY (x , y) = P(X ≤ x ,Y ≤ y).

For a continuous and differentiable FXY (x , y), the joint densityof X and Y is defined by

fXY (x , y) =∂2

∂x∂y[FXY (x , y)].

Notice that marginal distributions are

FX (x) = FXY (x ,∞),

FY (y) = FXY (∞, y).

Two Random VariablesThe marginal densities are obtained from

fX (x) =

∫ ∞−∞

fXY (x , y)dy ,

fY (y) =

∫ ∞−∞

fXY (x , y)dx .

Remark:

I X and Y are independent if:

FXY (x , y) = FX (x)FY (y)

or equally,fXY (x , y) = fX (x)fY (y).

I X and Y are independent identically distributed (i.i.d.) ifthey are independent and have the same distribution.

Two Random Variables - Correlation and CovarianceThe covariance CXY of X and Y is defined be

CXY ≡ 〈(X − 〈X 〉)(Y − 〈Y 〉)〉 = 〈XY 〉 − 〈X 〉〈Y 〉

The correlation coefficient ρXY of X and Y is defined be

ρXY ≡CXY

σXσY,

with the property that |ρXY | ≤ 1.Remark:

I X and Y are uncorrelated if:

CXY = 0 or ρXY = 0,

or equally,〈XY 〉 = 〈X 〉〈Y 〉.

I X and Y are orthogonal if 〈XY 〉 = 0.

Conclusions

I Random variables are the building block to studyuncertainties in a probabilistic framework

I In general, uncertain quantities cannot be described by ascalar quantities→ we introduce random vectors, andrandom fields.

Random Processes

Figure: Microstructures in a metal (From Professor H. Fujii, JWRI)

Random ProcessesA turbulent flow experiment

Figure: Three sample paths of a turbulence velocity component U at a fix location.From: S.B. Pope, Turbulent Flows.

Random ProcessesEarthquake time histories

Figure: Acceleration time history of the El Centro 1940 earthquake.

White noise v.s. non-white (colored) noise

Figure: A sample from a white noise process

Figure: A sample from a non-white noise process