steps in uncertainty quantification · i(q) j(q)⇢(q)dq = iji where i = e[2 i (q)] ... 2 9 29 81 5...
TRANSCRIPT
Steps in Uncertainty Quantification
Challenge:• How do we do uncertainty quantification for computationally expensive models?
• Example:
– We have a computational budget of 5000 model evaluations.
– Bayesian inference and uncertainty propagation require 120,000 evaluations.
Uncertainty Quantification ChallengesExample: MFC model – Fourth-order PDE
Capacitor probe
MFC Patch
Bayesian Inference: Took 6 days!Macro-Fiber Composite
Solution: Highly efficient surrogate models
Problem:
M = -c
E
I
@2w
@x
2 - c
D
I
@3w
@x
2@t
- [k1e(E ,�0)E + k2"irr
(E ,�0)]�MFC
(x)
⇢@2
w
@t
2 -@2
M
@x
2 = f
cE I1.2 ⇥ 10
5
PDE solutions
138
Surrogate Models: MotivationExample: Consider the heat equation
Notes:• Requires approximation of PDE in 3-D
• What would be a simple surrogate?
t
1 x , y , z
with the response
Boundary Conditions
Initial Conditions
y(q) =
Z1
0
Z 1
0
Z 1
0
Z 1
0u(t , x , y , z)dxdydzdt
@u
@t
=@2
u
@x
2 +@2
u
@y
2 +@2
u
@z
2 + f (q)
139
Surrogate Models: MotivationExample: Consider the heat equation
with the response
Boundary Conditions
Initial Conditions
y(q) =
Z1
0
Z 1
0
Z 1
0
Z 1
0u(t , x , y , z)dxdydzdt
@u
@t
=@2
u
@x
2 +@2
u
@y
2 +@2
u
@z
2 + f (q)
Question: How do you construct a polynomial surrogate?• Regression
• Interpolation
t
1 x , y , z
Surrogate: Quadraticys(q) = (q - 0.25)2 + 0.5
Surrogate ModelsRecall: Consider the model
Question: How do you construct a polynomial surrogate?• Interpolation
• RegressionM=7k=2
with the response
Boundary Conditions
Initial Conditions
y(q) =
Z1
0
Z 1
0
Z 1
0
Z 1
0u(t , x , y , z)dxdydzdt
@u
@t
=@2
u
@x
2 +@2
u
@y
2 +@2
u
@z
2 + f (q)
t
1 x , y , z
Surrogate: Quadraticys(q) = (q - 0.25)2 + 0.5
Surrogate Models
Question: How do we keep from fitting noise?• Akaike Information Criterion (AIC)
• Bayesian Information Criterion (AIC)
Likelihood:
M=7k=2
Minimize
Maximize
SSq =MX
m=1
[ym - ys(qm)]2
AIC = 2k - 2 log[⇡(y |q)]
BIC = k log(M)- 2 log[⇡(y |q)]
⇡(y |q) =1
(2⇡�2)M/2 e-SSq/2�2
142
Surrogate Models
Question: How do we keep from fitting noise?• Akaike Information Criterion (AIC)
• Bayesian Information Criterion (AIC)
Likelihood:
M=7k=2
Maximize
SSq =MX
m=1
[ym - ys(qm)]2
AIC = 2k - 2 log[⇡(y |q)]
BIC = k log(M)- 2 log[⇡(y |q)]
⇡(y |q) =1
(2⇡�2)M/2 e-SSq/2�2
Example: Exercise:• Construct a polynomial surrogate using the code
response_surface.m.
• What order seems appropriate?
y = exp(0.7q1
+ 0.3q2
)
!
q 2
q1
Minimize
Data-Fit ModelsNotes:• Often termed response surface models, emulators, meta-models.
• Constructed via interpolation or regression.
• Data can consist of high-fidelity simulations or experiments.
y = f(q)
Example: Steady-state Euler-Bernouilli beam model with PZT patch
Data: Displacement observations
Parameter:
YI
d
4w
dx
4 (x) = k
p
V�pzt
(x)
YI
144
Data-Fit ModelsExample: Steady-state Euler-Bernouilli beam model with PZT patch
Data: Displacement observationsParameter: YI Training points: 5000
Bayesian Inference
YI x
disp
lace
men
t
Polynomial surrogate: 6th order
YI
d
4w
dx
4 (x) = k
p
V�pzt
(x)
Data-Fit ModelsNotes:• Often termed response surface models, surrogates, emulators, meta-models.
• Rely on interpolation or regression.
• Data can consist of high-fidelity simulations or experiments.
• Common techniques: polynomial models, kriging (Gaussian process regression), orthogonal polynomials.
Statistical Model:
y = f(q)
Strategy: Consider high fidelity model
with M model evaluations
Surrogate:
Note: k(Q) orthogonal with respect to
inner product associated with pdf
e.g., Q ⇠ N(0, 1): Hermite polynomials
Q ⇠ U(-1, 1): Legendre polynomials
yK (Q) = fs(Q) =KX
k=0
↵k k(Q)
y = f (q)
ym = f (qm) , m = 1, ... , M
fs(q): Surrogate for f (q)
ym = fs(qm) + "m , m = 1, ... , M 146
Orthogonal Polynomial RepresentationsRepresentation:
yK (Q) =KX
k=0
↵k k(Q)
Note: 0(Q) = 1 implies that
E[ 0(Q)] = 1
E[ i(Q) j(Q)] =
Z
� i(q) j(q)⇢(q)dq
= �ij�i
where �i = E[ 2i (Q)]
Properties:
(i) E[yK (Q)] = ↵0
(ii) var[yK (Q)] =KX
k=1
↵2k�k
Note: Can be used for:• Uncertainty propagation
• Sobol-based global sensitivity analysis
Issue:• Stochastic Galerkin techniques (Polynomial Chaos Expansion – PCE)
• Nonintrusive PCE (Discrete projection)
• Stochastic collocation
• Regression-based methods with sparsity control (Lasso)
How does one compute ↵k , k = 0, ... , K ?
Note: Methods nonintrusive and treat code as blackbox. 147
Orthogonal Polynomial RepresentationsNonintrusive PCE:
to obtain
Take weighted inner product of y(q) =P1
k=0
↵k k(q)
↵k =1�k
Z
�y(q) k(q)⇢(q)dq
Note: Sample points
↵k ⇡ 1�k
RX
r=1
y(qr ) k(qr )wr
Note:(i) Low-dimensional: Tensored 1-D quadrature rules – e.g., Gaussian
(ii) Moderate-dimensional: Sparse grid (Smolyak) techniques
(iii) High-dimensional: Monte Carlo or quasi-Monte Carlo (QMC) techniques
Regression-Based Methods with Sparsity Control (Lasso): Solve
Quadrature:
d = [y(q1) , ... , y(qm)]
{qm}Mm=1e.g., SPGL1
• MATLAB Solver for large-scale sparse reconstruction
⇤ 2 RM⇥(K+1) where ⇤jk = k(qj)
min
↵2RK+1
k⇤↵- dk2
subject to
KX
k=0
|↵k | 6 ⌧
148
Stochastic CollocationStrategy: Consider high fidelity model
with M model evaluations
Collocation Surrogate:
Note: Result:
ym
Lm(q) =MY
j=0j 6=m
q - q j
q m - q j =(q - q 1) · · · (q - q m-1)(q - q m+1) · · · (q - q M)
(q m - q 1) · · · (q m - q m-1)(q m - q m+1) · · · (q m - q M)
Lm(q j) = �jm =
�0 , j 6= m1 , j = m
y = f (q)
ym = f (qm) , m = 1, ... , M
where Lm(q) is a Lagrange polynomial, which in 1-D, is represented by
Y M(q) =MX
m=1
ymLm(q)
Y M(qm) = ym
149
Orthogonal Polynomial Methods for PDE
Evolution Model: e.g., thermal-hydraulic equations
@u
@t
= N(u, Q) + F (Q) , x 2 D, t 2 [0,1)
B(u, Q) = G(Q) , x 2 @D, t 2 [0,1)
u(0, x , Q) = I(Q) , x 2 D
Weak Formulation: For all v 2 VZ
D
@u
@t
vdx +
Z
D
N(u, Q)S(v)dx =
Z
D
F (Q)vdx
Response: y(t , x) =
Z
�u(t , x , q)⇢(q)dq
Example: q = ↵
@u
@t
= ↵@2
u
@x
2
y(t , 0) = u(t , L) = 0
u(0, x) = u0(x)
For all v 2 H1
0
(0, L)Z
L
0
@u
@t
vdx + ↵
ZL
0
@u
@x
dv
dx
= 0
Representation:
u
K (t , x , Q) =KX
k=0
u
k
(t , x) k
(Q)
=KX
k=0
JX
j=1
u
jk
(t)�j
(x) k
(Q)
u
k
(t , x) ⇡ 1�
k
RX
r=1
u(t , x , q
r ) k
(qr )wr
Discrete Projection:
e.g., Finite elements150
Surrogate Models – Grid Choice
Example:
−1 −0.5 0 0.5 1−0.2
0
0.2
0.4
0.6
0.8
1
1.2
q
f(q)
Consider the Runge function f (q) = 1
1+25q2
with points
qj = -1 + (j - 1)2M
, j = 1, ... , M
151
Surrogate Models – Grid Choice
Example:
−1 −0.5 0 0.5 1−0.2
0
0.2
0.4
0.6
0.8
1
1.2
q
f(q)
−1 −0.5 0 0.5 1−80
−60
−40
−20
0
20
q
f(q)
Consider the Runge function f (q) = 1
1+25q2
with points
qj = -1 + (j - 1)2M
, j = 1, ... , M
152
Surrogate Models – Grid Choice
Example:
−1 −0.5 0 0.5 1−0.2
0
0.2
0.4
0.6
0.8
1
1.2
q
f(q)
−1 −0.5 0 0.5 10
0.2
0.4
0.6
0.8
1
q
f(q)
−1 −0.5 0 0.5 1−80
−60
−40
−20
0
20
q
f(q)
Consider the Runge function f (q) = 1
1+25q2
with points
qj = -1 + (j - 1)2M
, j = 1, ... , M qj = - cos
⇡(j - 1)
M - 1
, j = 1, ... , M
153
Surrogate Models – Grid Choice
Example:
−1 −0.5 0 0.5 1−0.2
0
0.2
0.4
0.6
0.8
1
1.2
q
f(q)
−1 −0.5 0 0.5 10
0.2
0.4
0.6
0.8
1
q
f(q)
−1 −0.5 0 0.5 1−80
−60
−40
−20
0
20
q
f(q)
−1 −0.5 0 0.5 10
0.2
0.4
0.6
0.8
1
q
f(q)
Consider the Runge function f (q) = 1
1+25q2
with points
qj = -1 + (j - 1)2M
, j = 1, ... , M qj = - cos
⇡(j - 1)
M - 1
, j = 1, ... , M
154
Sparse Grid TechniquesTensored Grids: Exponential growth Sparse Grids: Same accuracy
p R` Sparse Grid R Tensored Grid R = (R`)p
2 9 29 81
5 9 241 59,049
10 9 1581 > 3⇥ 109
50 9 171,901 > 5⇥ 1047
100 9 1,353,801 > 2⇥ 1095
155
Surrogate Construction: Nuclear Power Plant Design
Subchannel Code (COBRA-TF): 33 VUQ parameters reduced to 5 using SA
Surrogate: Total pressure drop
• Kriging (GP) emulator constructed using 50 COBRA-TF runs perturbing 5 active inputs.
• Use remaining computational budget to evaluate quality of surrogate using post-processed Dakota outputs.
●●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
1.10 1.15 1.20 1.25
1.10
1.15
1.20
1.25
Calculated Total Pressure Drop
Pred
icte
d To
tal P
ress
ure
Dro
p
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
1.10 1.15 1.20 1.25
−10
12
Calculated Total Pressure Drop
Stan
dard
ized
Res
idua
l Tot
al P
ress
ure
Dro
p
Pred
icte
d To
tal P
ress
ure
Dro
p
Calculated Total Pressure Drop Theoretical Quantities
Sam
ple
Qua
ntiti
es
Out-of-Sample Validation
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
−2 −1 0 1 2
−2−1
01
Leave−one−out Cross Validation
Theoretical Quantiles
Sam
ple
Qua
ntile
s
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
−2 −1 0 1 2
−10
12
Out−of−sample Validation
Theoretical Quantiles
Sam
ple
Qua
ntile
s
156
Example: SIR Cholera ModelModel:
dSdt
= bN - �LSBL
L + BL- �HS
BH
H + BH- bS
dIdt
= �LSBL
L + BL+ �HS
BH
H + BH- (�+ b)I
dRdt
= �I - bR
dBH
dt= ⇠I - �BH
dBL
dt= �BH - �BL
Model Parameter Symbol Units ValuesRate of drinking BL cholera �L
1week 1.5
Rate of drinking BH cholera �H1
week 7.5 (⇤)
BL cholera carrying capacity L# bacteria
m` 106
BH cholera carrying capacity H# bacteria
m`L700
Human birth and death rate b 1week
11560
Rate of decay from BH to BL � 1week
1685
Rate at which infectious individuals ⇠ # bacteria# individuals·m`·week 70
spread BH bacteria to waterDeath rate of BL cholera � 1
week7
30Rate of recovery from cholera � 1
week75
δ
BH
BL
βH
βL
I RSγ
ξ
χ
157
�L �HL b � ⇠ �L �
0
0.1
0.2
0.3
0.4
0.5
parameter
generalizedSobolindex
MC (Nmc = 10
4)
Alg 1 (Nquad = 609)
Alg 1 (Nquad = 2177)
Alg 2 (Nmc = 150/Nkl = 2)
Alg 2 (Nmc = 150/Nkl = 5)
Alg 2 (Nmc = 150/Nkl = 15)
Example: SIR Cholera ModelStrategy: Employed collocation and discrete projection with sparse grids to compute time-dependent global sensitivity indices.
Conclusion: Sensitive indices�: Recovery rate
�H : Rate of drinking BH cholera
L: BL carrying capacity; Note H = L/700
⇠: Rate at which BH bacteria spread
10
�110
010
110
20
0.2
0.4
0.6
0.8
1
time [weeks]
totalindices
�L
�HL
b�⇠�L�
10
�110
010
110
20
0.2
0.4
0.6
0.8
1
time [weeks]
generalizedtotalindices
�L
�HL
b�⇠�L�
δ
BH
BL
βH
βL
I RSγ
ξ
χ
Example: SIR Cholera ModelModel:
dSdt
= bN - �LSBL
L + BL- �HS
BH
H + BH- bS
dIdt
= �LSBL
L + BL+ �HS
BH
H + BH- (�+ b)I
dRdt
= �I - bR
dBH
dt= ⇠I - �BH
dBL
dt= �BH - �BL
Model Parameter Symbol Units ValuesRate of drinking BL cholera �L
1week 1.5
Rate of drinking BH cholera �H1
week 7.5 (⇤)
BL cholera carrying capacity L# bacteria
m` 106
BH cholera carrying capacity H# bacteria
m`L700
Human birth and death rate b 1week
11560
Rate of decay from BH to BL � 1week
1685
Rate at which infectious individuals ⇠ # bacteria# individuals·m`·week 70
spread BH bacteria to waterDeath rate of BL cholera � 1
week7
30Rate of recovery from cholera � 1
week75
159
Steps in Uncertainty Quantification
160
6. Quantification of Model Discrepancy – Thin Beam“Essentially all models are wrong, but some are useful” George E.P. Box
with
Example: Thin beam driven by PZT patches
Euler-Bernoulli Model:
Statistical Model:
Note: 7 parameters, 32 states
For all � 2 V
ZL
0
⇢(x)
@2w
@t
2 + �@w
dt
��dx +
ZL
0
YI(x)
@2w
@x
2 + cI(x)@3
w
@x
2@t
�� 00
dx
= k
p
V (t)
Zx2
x1
� 00dx
⇢(x) = ⇢hb + ⇢p
h
p
b
p
�p
(x) , YI(x) = YI + Y
p
I
p
�p
(x)
cI(x) = cI + c
p
I
p
�p
(x)y(t
i
, q) = w(ti
, x , q)
ti
YiYi = y(ti , q) + "i 161
Quantification of Model Discrepancy – Thin BeamExample: Good model fit
Note: Observation errors not iid
Model Fit to Data
0 20 40 60 80 100 1200
0.2
0.4
0.6
0.8
1
x 10−7
Frequency
Fre
quency
Conte
nt
Mathematical Model y(ti;q)
Data
Reference: Additive observation errors
Yi = y(ti , q) + "i
Yi = y(ti , q) + �(ti , eq) + "i
• M.C. Kennedy and A. O’Hagan, Journal of the Royal Statistical Society, Series B, 2001.
Quantification of Model Discrepancy – Thin BeamExample: Good model fit
Problem: Observation errors not iid
Result: Prediction intervals wrong
Prediction Intervals and Data
Model Fit to Data
Approaches: • GP Model: Inaccurate for extrapolation• Control-based approaches: difficult to extrapolate.• Problem: correct physics or biology required for extrapolation!
Quantification of Model Discrepancy – Thin BeamProblem: Measurement errors not iid
Result: Prediction intervals wrong
Prediction Intervals and Data
Model Fit to Data
One Approach: • Determine components of model you trust (e.g., conservation laws) and don’t trust (e.g., closure relations). Embed uncertainty into latter.• T. Oliver, G. Terejanu, C.S. Simmons, R.D. Moser, Comput Meth Appl Mech Eng, 2015.
2018-19 SAMSI Program: Model Uncertainty: Mathematical and Statistical (MUMS)
Quantification of Model Discrepancy – Thin Beam
Our Solution: “Optimize” calibration interval
• Use damping/frequency domain results to guide.
0.75 0.8 0.85 0.9 0.95 1−60
−40
−20
0
20
40
60
Time (s)
Dis
pla
cem
en
t (µ
m)
2 2.1 2.2 2.3 2.4 2.5−50
0
50
Time (s)
Dis
pla
cem
en
t (µ
m)
3 3.1 3.2 3.3 3.4 3.5−40
−30
−20
−10
0
10
20
30
40
Time (s)
Dis
pla
cem
en
t (µ
m)
0 1 2 3−30
−20
−10
0
10
20
30
Time (s)
Dis
pla
cem
ent (µ
m)
0 1 2 3−40
−30
−20
−10
0
10
20
30
Time (s)
Dis
pla
cem
en
t (µ
m)
Note: We have substantially extended calibration regime.Calibrate on [0,1] Calibrate on [0.25,1.25]
Concluding RemarksNotes:• UQ requires a synergy between engineering, statistics, and applied mathematics.
• Model calibration, model selection, uncertainty propagation and experimental design are natural in a Bayesian framework.
• Goal is to predict model responses with quantified and reduced uncertainties.
• Parameter selection is critical to isolate identifiable and influential parameters.
• Surrogate models critical for computationally intensive simulation codes.
• Codes and packages: Sandia Dakota, R, MATLAB, Python, nanoHUB.
• Prediction is very difficult, especially if it’s about the future, Niels Bohr.
2 2.1 2.2 2.3 2.4 2.5−50
0
50
Time (s)
Dis
pla
cem
en
t (µ
m)
References
Books:
• R.C. Smith, Uncertainty Quantification: Theory, Implementation, and Applications, SIAM, Philadelphia, 2014.
Incomplete References:
Orthogonal Polynomial Methods and High-Dimensional Approximation Theory:• M. Babuška, F. Nobile and R. Tempone, Numer. Math., 2005
• A. Cohen, R. De Vore and C. Schwab, Analysis and Applications, 2011
• M. Gunzburger, C.G. Webster and G. Zhang, Acta Numerica, 2014.
• O.P. Le Maître and O.M. Knio, Spectral Methods for Uncertainty Quantification, 2010
• S., Uncertainty Quantification: Theory, Implementation, and Applications, 2014.
• A.L. Teckentrup, P. Jantsch, M. Gunzburger and C.G. Webster, SIAM/ASA J. Uncert. Quant., 2015
• D. Xiu, Numerical Methods for Stochastic Computations: A Spectral Method Approach, 2010.
167
Incomplete References:
Sparse Grids:• H-J. Bungartz and M. Griebel, Acta Numerica, 2004
• M. Gunzburger et al., Water Resources Research, 2013.
• F. Nobile, R. Tempone and C.G. Webster, SIAM J. Num. Anal., 2008
Compressed Sensing:• A. Chkifra, N. Dexter, H. Tran and C.G. Webster, Mathematics of Computation, Submitted
References
Infinite-Dimensional Bayesian Inference: • A.M. Stuart, Acta Numerica 2010
• T. Bui-Thanh et al., SIAM J. Sci. Comput., 2013
• S.J. Vollmer, SIAM/ASA J. Uncertainty Quantification, 2015.
• T. Bui-Thanh and Q. Nguyen, Inverse Problems and Imaging, 2016, 168