a dynamically bi-orthogonal method for time...
TRANSCRIPT
A Dynamically Bi-Orthogonal Method for Time-Dependent StochasticPartial Differential Equations I: Derivation and Algorithms
Mulin Chenga, Thomas Y. Houa,∗, Zhiwen Zhanga
aComputing and Mathematical Sciences, California Institute of Technology, Pasadena, CA 91125
Abstract
We propose a dynamically bi-orthogonal method (DyBO) to solve time dependent stochastic partial dif-ferential equations (SPDEs). The objective of our method is to exploit some intrinsic sparse structurein the stochastic solution by constructing the sparsest representation of the stochastic solution via a bi-orthogonal basis. It is well-known that the Karhunen-Loeve expansion (KLE) minimizes the total meansquared error and gives the sparsest representation of stochastic solutions. However, the computation of theKL expansion could be quite expensive since we need to form a covariance matrix and solve a large-scaleeigenvalue problem. The main contribution of this paper is that we derive an equivalent system that governsthe evolution of the spatial and stochastic basis in the KL expansion. Unlike other reduced model meth-ods, our method constructs the reduced basis on-the-fly without the need to form the covariance matrixor to compute its eigendecomposition. In the first part of our paper, we introduce the derivation of thedynamically bi-orthogonal formulation for SPDEs, discuss several theoretical issues, such as the dynamicbi-orthogonality preservation and some preliminary error analysis of the DyBO method. We also give somenumerical implementation details of the DyBO methods, including the representation of stochastic basisand techniques to deal with eigenvalue crossing. In the second part of our paper [11], we will present anadaptive strategy to dynamically remove or add modes, perform a detailed complexity analysis, and discussvarious generalizations of this approach. An extensive range of numerical experiments will be provided inboth parts to demonstrate the effectiveness of the DyBO method.
Keywords: Stochastic partial differential equations, Karhunen-Loeve expansion, uncertaintyquantification, bi-orthogonality, reduced-order model.
1. Introduction
Uncertainty arises in many complex real-world problems of physical and engineering interests, such aswave, heat and pollution propagation through random media [31, 19, 17, 53] and flow driven by stochasticforces [26, 28, 50, 42, 37]. Additional examples can be found in other branches of science and engineer-ing, such as geosciences, statistical mechanics, meteorology, biology, finance, and social science. Stochasticpartial differential equations (SPDEs) have played an important role in our investigation of uncertaintyquantification (UQ). However, it is very challenging to solve stochastic PDEs efficiently due to the largedimensionality of stochastic solutions. Although many numerical methods have been proposed in the liter-ature (see e.g. [52, 9, 12, 13, 23, 30, 38, 54, 4, 5, 55, 36, 51, 28, 48, 1, 39, 24, 46, 14, 34, 15, 16, 40, 44]), it isstill very expensive to solve a time dependent nonlinear stochastic PDEs with high dimensional stochasticinput variables. This limits our ability to attack some challenging real-world applications.
In this paper, we propose a dynamically bi-orthogonal method (DyBO) to solve time-dependent stochasticpartial differential equations. The dynamic bi-orthogonality condition that we impose on DyBO is closely
∗Corresponding authorEmail addresses: [email protected] (Mulin Cheng), [email protected] (Thomas Y. Hou), [email protected]
(Zhiwen Zhang)
Preprint submitted to Elsevier Friday 8th February, 2013
related to the Karhunen-Loeve expansion (KLE) [29, 32]. The KL expansion has received increasing attentionin many fields, such as statistics, image processing, and uncertainty quantification. It yields the best basis inthe sense that it minimizes the total mean squared error and gives the sparsest representation of stochasticsolutions. However, computing KLE could be quite expensive since it involves forming a covariance matrixand solving the associated large-scale eigenvalue problem. One of the attractive features of our DyBO methodis that we construct the sparsest bi-orthogonal basis on-the-fly without the need to form the covariancematrix or to compute its eigendecomposition.
We consider the following time-dependent stochastic partial differential equations:
∂u
∂t(x, t, ω) = Lu(x, t, ω), x ∈ D ⊂ Rd, ω ∈ Ω, t ∈ [0, T ], (1a)
u(x, 0, ω) = u0(x, ω), x ∈ D, ω ∈ Ω, (1b)
B(u(x, t, ω)) = h(x, t, ω), x ∈ ∂D, ω ∈ Ω, (1c)
where L is a differential operator and may contain random coefficients and/or stochastic forces and B is aboundary operator. The randomness may also enter the system through the initial condition u0 and/or theboundary conditions B.
We assume the stochastic solution u(x, t, ω) of the system (1) is a second-order stochastic process, i.e.,u(·, t, ·) ∈ L2 (D × Ω). The solution u(x, t, ω) can be represented by its KL expansion as follows:
u(x, t, ω) = u(x, t) +
∞∑i=1
ui(x, t)Yi(ω, t), (2)
where u(x, t) = E [u(x, t, ω)], ui’s are the eigenfunctions of the associated covariance function Covu(x, y) =E [(u(x, t, ω)− u(x, t))(u(y, t, ω)− u(y, t))]. Thus ui are orthogonal to each other, i.e., 〈ui, uj〉 = λiδij , whereλi are the corresponding eigenvalues of the covariance function. The random variables Yi have zero-meanand are mutually orthogonal, i.e. E [YiYj ] = δij . They are defined as follows:
Yi(ω, t) =1
λi(t)
∫D
(u(x, t, ω)− u(x, t))ui(x, t) dx, for i = 1, 2, 3, · · · .
For many physical and engineering problems, the dimension of the input stochastic variables may appearto be high, but the effective dimension of the stochastic solution may be low due to the rapid decay ofthe eigenvalues λi [48]. For this type of problems, the KLE provides the sparest representation of thesolution through an optimal set of bi-orthogonal spatial and stochastic basis. Since this bi-orthogonal basisis constructed at each time, both the spatial basis ui and the stochastic basis Yi are time-dependent topreserve bi-orthogonality. This is very different from other reduced-order basis in which either the spatialor the random basis is time-dependent, but not both of them are time-dependent. The traditional wayto construct the KL expansion is to first solve SPDE by some conventional method, and then form thecovariance function to construct the KL expansion. As we mentioned earlier, this post-processing approachis very expensive which would offset the benefit of giving a sparsest representation of the stochastic solution.
The main objective of this paper is to derive a well-posed system that governs the dynamic evolutionof the KL expansion without the need of forming the covariance function explicitly. To illustrate the mainidea, we assume that the eigenvalues λi decay rapidly and an m-term truncation in the KL expansion wouldgive an accurate approximation of the stochastic solution. We substitute the m-term truncated expansion(2), denoted as u, into the stochastic PDE (1a). After projecting the resulting equation into the spatial andstochastic basis and using the bi-orthogonality of the basis, we obtain
∂u
∂t= E [Lu] , (3a)
∂U
∂t= UΛ−1
U
⟨UT ,
∂U
∂t
⟩+GU(u,U,Y), (3b)
dY
dt= YE
[YT dY
dt
]+GY(u,U,Y), (3c)
2
where E [·] is the expectation, 〈f, g〉 =∫f(x)g(x)dx, U(x, t) = (u1(x, t), u2(x, t), ..., um(x, t)), Y(ω, t) =
(Y1(ω, t), Y2(ω, t), ..., Ym(ω, t)), u = u + UYT , ΛU = diag(⟨UT , U
⟩), GU(u,U,Y) and GY(u,U,Y) are
defined in (12a) and (12b) respectively. The initial condition of (3) is given by the KL expansion of u0(x, ω).A close inspection shows that the evolution system (3b) and (3c) does not determine the time derivative
∂U∂t or dY
dt uniquely. In fact, (3b) only determines the projection of ∂U∂t into the orthogonal complement
set of U uniquely, which is GU. Similarly, (3c) only determines the projection of dYdt into the orthogonal
complement set of Y uniquely, which is GY. Thus, the evolution system (3) can be rewritten as follows:
∂u
∂t= E [Lu] , (4a)
∂U
∂t= UC +GU(u,U,Y), (4b)
dY
dt= YD +GY(u,U,Y), (4c)
where C and D are m × m matrices which represent the projection coefficients of ∂U∂t and dY
dt into Uand Y respectively. These two matrices characterize the large degrees of freedom in choosing the spatialand stochastic basis dynamically. They cannot be chosen independently. Instead, they must satisfy acompatibility condition to be consistent with the original SPDE and satisfy the bi-orthogonality conditionof the spatial and the stochastic basis. By introducing two types of antisymmetric operators to enforce thebi-orthogonality condition, we derive two constraints for C and D. When combining these two constraintswith the compatibility condition, we show that these two matrices can be determined uniquely providedthat the eigenvalues of the covariance matrix are distinct.
We prove rigorously that the system of governing equations indeed preserve the bi-orthogonality condition
dynamically. More precisely, if we write χ(t) = (〈ui, uj〉i>j) as a column vector in Rm(m−1)
2 ×1, then χ(t)
satisfies a linear ODE system dχ(t)dt =W(t)χ(t) for some anti-symmetric matrixW(t). The fact thatW(t) is
anti-symmetric is important because this implies that the dynamic preservation of bi-orthogonality conditionis very stable to perturbation and any deviation from the bi-orthogonal form will not be amplified.
The breakdown in deriving a well-posed system for the KL expansion when two distinct eigenvaluescoincide dynamically is associated with additional degrees of freedom in choosing orthogonal eigenvectors. Toovercome this difficulty, we design a strategy to handle eigenvalue crossings. When two distinct eigenvaluesbecome close, we temporarily freeze the stochastic basis and evolve only the spatial basis. By doing so, westill maintain the dynamic orthogonality of the stochastic basis, which allows us to compute the covariancematrix relatively easily. After the two eigenvalues separate in time, we can reinitialize the DyBO methodby reconstructing the KL expansion without forming the covariance matrix. Similarly, we can freeze thespatial basis and evolve the stochastic basis for a short time until the two eigenvalues are separated.
We remark that the low-dimensional or sparse structures of SPDE solutions have been explored partiallyin some methods, such as the proper orthogonal decomposition (POD) methods [3, 49, 50], reduced-basis(RB) methods [7, 35, 45, 21], and dynamical orthogonal method (DO) [46, 47]. Our method shares somecommon feature with the dynamically orthogonal method since both methods use time-dependent spatialand stochastic basis. An important ingredient in POD and RB methods is how to choose the reduced basisfrom some limited information of the solution that we learn offline. The main difference between our methodand other reduced basis methods is that we bypass the step of learning the basis offline, which in generalcan be quite expensive. We construct the reduced basis on the fly as we solve the SPDE.
We have performed some preliminary error analysis for our method. In the case when the differentialoperator is linear and deterministic and the randomness enters through the initial condition or forcing,we show that the error is due to the projection of the initial condition or the random forcing into the bi-orthogonal basis. In the special case when the initial condition has exactly m modes in the KL expansion,the m-term DyBO method will reproduce the exact solution without any error. For general nonlinearstochastic PDEs, the number of effective modes in the KL expansion will increase dynamically due tononlinear interaction. In this case, even if the initial condition has an exactly m-mode KL expansion, wemust dynamically increase modes to maintain the accuracy of our computation.
3
0 10 20 30 40 50 6010
−3
10−2
10−1
100
DyBO ‖wi‖L2
√λi
gPC ‖wα‖L2
0 10 20 30 40 50 6010
−3
10−2
10−1
100
gPC
DyBO
∼ e−0.62m
∼ e−0.065NP
Figure 1: Stochastically forced Burgers equations computed by DyBO and gPC. Left: (Sorted) Energyspectrum of gPC and DyBO solutions. Right: Relative errors of STD, DyBO (e−0.62m) vs gPC (e−0.065Np).
We have developed an effective adaptive strategy for DyBO to add or remove modes dynamically tomaintain a desirable accuracy. If the magnitude of the last mode falls below a prescribed tolerance, we canremove this mode dynamically. On the other hand, when the magnitude of the last mode rises above a certaintolerance, we need to add a new mode pair. The method we propose to add new modes dynamically is touse the DyBO solution as the initial condition and compute a DyBO solution and a generalized PolynomialChaos (gPC) solution respectively for a short time. We then compute the KL expansion of the differencebetween the DyBO solution and the gPC solution. We add the largest mode from the KL expansion of thedifference to our DyBO method. This method works quite effectively. More details will be reported in thesecond part of our paper [11].
As we will demonstrate in our paper, the DyBO method could offer accurate numerical solutions toSPDEs with significant computational saving over traditional stochastic methods. We have performed nu-merical experiments for 1D Burgers equations, 2D incompressible Navier-Stokes equations, and Boussinesqequations with approximated Brownian motion forcing. In all these cases, we observe considerable com-putational saving. The specific rate of saving will depend on how we discretize the stochastic ordinarydifferential equations (SODEs) governing the stochastic basis. We can use three methods to discretize theSODEs, i.e. (i) Monte Carlo methods (MC), (ii) generalized stochastic collocation methods (gSC), (iii)generalized polynomial chaos methods (gPC). In the numerical experiments we present in this paper, we usegPC to discretize the SODE. Our numerical results show that the DyBO method gives much faster decayrate for its error compared with that of the gPC method. To illustrate this point, we show the sorted energyspectrum of the solution of the stochastically forced Burgers equation computed by our DyBO method andthat of the gPC method respectively in Figure 1. We can see that the eigenvalues of the covariance matrixdecay much faster than the corresponding sorted energy spectrum of the gPC solution, indicating the effec-tive dimension reduction induced by the bi-orthogonal basis. If we denote m as the number of modes usedby DyBO and Np the number of modes used by gPC, our complexity analysis shows that the ratio betweenthe computational complexities of DyBO and gPC is roughly of order O(m/Np)
3 for a two-dimensionalquadratic nonlinear SPDE. Typically m is much smaller than Np, as illustrated in Figure 1. More detailscan be found in part II of our paper where we also discuss the parallel implementation of DyBO [11].
This paper is organized as follows. In Section 2, we provide the detailed derivation of the DyBO for-mulation for a time-dependent SPDE. Some important theoretic aspects of the DyBO formulation, such asbi-orthogonality preservation and the consistency between the DyBO formulation and the original SPDEare discussed in Section 3. In Section 4, we discuss some detailed numerical implementations of the DyBOmethod, such as the representation of the stochastic basis Y and a strategy to deal with eigenvalue crossings.Three numerical examples are provided in Section 5 to demonstrate computational advantage of our DyBOmethod. Finally, some remarks will be made in Section 6.
4
2. Derivation of dynamically bi-orthogonal formulation
In this section, we introduce the derivation of our dynamically bi-orthogonal method. For the ease offuture derivations, we use extensively vector and matrix notations.
U(x, t) = (u1(x, t), u2(x, t), ..., um(x, t)) , U(x, t) = (um+1(x, t), um+2(x, t), ...) ,
Y(ω, t) = (Y1(ω, t), Y2(ω, t), ..., Ym(ω, t)) , Y(ω, t) = (Ym+1(ω, t), Ym+2(ω, t), ...) .
Correspondingly, E[YTY] and⟨UT , U
⟩are m-by-m matrices and the bi-orthogonality condition can be
rewritten compactly as
E[YTY
](t) = (E [YiYj ]) = I ∈ Rm×m, (5a)⟨
UT , U⟩
(t) = (〈ui, uj〉) = ΛU ∈ Rm×m, (5b)
where ΛU = diag(⟨UT , U
⟩) = (λiδij)m×m. Therefore, the KL expansion (2) of u at some fixed time t reads
u = u+ UYT + UYT . (6)
Furthermore, we assume that the solution of the SPDE (1) enjoys a low-dimensional structure in the sense ofKLE, or precisely, the eigenvalue spectrum λi(t)∞i=1 decays fast enough, allowing for a good approximationwith a truncated KL expansion (2). We denote by u as the m-term truncation of the KL expansion of u,
u = u+ UYT . (7)
We introduce the following orthogonal complementary operators with respect to the orthogonal basis U inL2 (D) or Y in L2 (Ω) to simplify notations,
ΠU(v) = v −UΛ−1U
⟨UT , v
⟩for v(x) ∈ L2 (D) ,
ΠY(Z) = Z −YE[YTZ
]for Z(ω) ∈ L2 (Ω) .
We also define an anti-symmetrization operator Q : Rk×k → Rk×k and a partial anti-symmetrizationoperator Q : Rk×k → Rk×k, which are defined as follows:
Q(A) =1
2
(A−AT
), Q(A) =
1
2
(A−AT
)+ diag(A),
where diag(A) denotes a diagonal matrix whose diagonal entries are equal to those of matrix A.In the DyBO formulation it is crucial to dynamically preserve the bi-orthogonality of the spatial basis U
and the stochastic basis Y. We begin the derivation by substituting the KLE (2) into the SPDE (1a) andget
∂u
∂t+∂U
∂tYT + U
dYT
dt= Lu+ Lu− Lu −
∂U
∂tYT + U
dYT
dt
.
If the eigenvalues in the KL expansion decay fast enough and the differential operator is stable, the last twoterms on the right hand side will be small, so we can drop them and obtain the starting point for derivingour DyBO method,
∂u
∂t+∂U
∂tYT + U
dYT
dt= Lu. (9)
Taking expectations on both sides of Eq. (9) and taking into account the fact that Yi’s are zero-mean randomvariables, we have
∂u
∂t= E [Lu] ,
5
which gives the evolution equation for the mean of the solution u. Multiplying both sides of Eq. (9) by Yfrom the right and taking expectations, we obtain
∂U
∂tE[YTY
]+ UE
[dYT
dtY
]= E
[(Lu− ∂u
∂t)Y
].
After using the orthogonality of the stochastic basis Y, we have
∂U
∂t= E
[LuY
]−UE
[dYT
dtY
],
where Lu = Lu − E [Lu]. Similarly we obtain an evolution equation for dYdt . Therefore, this gives rise to
the following evolution system:
∂u
∂t= E [Lu] , (10a)
∂U
∂t= E
[LuY
]−UE
[dYT
dtY
], (10b)
dY
dtΛU =
⟨Lu, U
⟩−Y
⟨∂UT
∂t, U
⟩. (10c)
However, the above system does not give explicit evolution equations for dYdt and ∂U
∂t . To further simplifythe evolution system, we substitute Eq. (10c) into Eq. (10b) and get
∂U
∂t= E
[LuY
]−UΛ−1
U
⟨UT , E
[LuY
]⟩+ UΛ−1
U
⟨UT ,
∂U
∂t
⟩,
where we have used the orthogonality of the stochastic basis, i.e., E[YTY
]= I. Similar steps can be
repeated by substituting Eq. (10b) into Eq. (10c). Thus, the system (10) can be re-written as
∂u
∂t= E [Lu] , (11a)
∂U
∂t= UΛ−1
U
⟨UT ,
∂U
∂t
⟩+GU(u,U,Y), (11b)
dY
dt= YE
[YT dY
dt
]+GY(u,U,Y), (11c)
where we have used the orthogonality condition of the spatial and stochastic modes, and
GU(u,U,Y) = ΠU
(E[LuY
])= E
[LuY
]−UΛ−1
U
⟨UT , E
[LuY
]⟩, (12a)
GY(u,U,Y) = ΠY
(⟨Lu, U
⟩)Λ−1
U =⟨Lu, U
⟩Λ−1
U −Y⟨E[LuYT
], U⟩
Λ−1U . (12b)
Note that GU is the projection of E[LuY
]into the orthogonal complementary set of U and GY is the
projection of⟨Lu, U
⟩into the orthogonal complementary set of Y. Thus, we have
GU(u,U,Y) ⊥ spanU in L2 (D) , GY(u,U,Y) ⊥ spanY in L2 (Ω) .
The above observation will be crucial in the later derivations.We note that the system (11b) and (11c) does not determine the time derivative ∂U
∂t or dYdt uniquely.
In fact, if ∂U∂t is a solution of (11b), then ∂U
∂t + UC for any m ×m matrix C is also a solution of (11b).
6
Similar statement can be made for dYdt . In Appendix A, we will show that the evolution system (3) can be
rewritten as follows:
∂u
∂t= E [Lu] , (13a)
∂U
∂t= UC +GU(u,U,Y), (13b)
dY
dt= YD +GY(u,U,Y), (13c)
where C and D are m × m matrices which represent the projection coefficients of ∂U∂t and dY
dt into Uand Y respectively. Moreover, we show that by enforcing the bi-orthogonality condition of the spatial andstochastic basis and a compatibility condition with the original SPDE, these two matrices can be determineduniquely provided that the eigenvalues of the covariance matrix are distinct. The results are summarized inthe following theorem.
Theorem 2.1 (Solvability of matrices C and D). If 〈ui, ui〉 6= 〈uj , uj〉 for i 6= j, i, j = 1, 2, . . . ,m, them-by-m matrices C and D can be solved uniquely from the following linear system
C−Λ−1U Q (ΛUC) = 0, (14a)
D−Q (D) = 0, (14b)
DT + C = G∗(u,U,Y), (14c)
where G∗(u,U,Y) = Λ−1U
⟨UT , E
[LuY
]⟩∈ Rm×m. The solutions are given entry-wisely as follows:
Cii = G∗ii, (15a)
Cij =‖uj‖2L2(D)
‖uj‖2L2(D) − ‖ui‖2L2(D)
(G∗ij +G∗ji) , for i 6= j, (15b)
Dii = 0, (15c)
Dij =1
‖uj‖2L2(D) − ‖ui‖2L2(D)
(‖uj‖2L2(D)G∗ji + ‖ui‖2L2(D)G∗ij
), for i 6= j. (15d)
To summarize the above discussion, the DyBO formulation of SPDE (1) results in a set of explicitequations for all the unknowns quantities. In particular, we reformulate the original SPDE (1) into a systemof m stochastic ordinary differential equations for the stochastic basis Y coupled with m+ 1 deterministicPDEs for the mean solution u and spatial basis U. Further, from the definition of G∗ given in Theorem 2.1,we can rewrite GU and GY as
GU = E[LuY
]−UG∗, GY =
⟨Lu, U
⟩Λ−1
U −YGT∗ .
From the above identities, we can rewrite the DyBO formulation (13) as follows:
∂u
∂t= E [Lu] , (16a)
∂U
∂t= −UDT + E
[LuY
], (16b)
dY
dt= −YCT +
⟨Lu, U
⟩Λ−1
U , (16c)
where we have used the compatibility condition Eq.(14c) for C and D, i.e., DT + C = G∗. The initialconditions of u, U and Y are given by the KL expansion of u0(x, ω) in Eq.(1b). We can derive the
7
boundary condition of u and the spatial basis U by taking expectation on the boundary condition. Forexample, if B is a linear deterministic differential operator, we have B(u(x, t, ω))|x∈∂D = E [h(x, t, ω)] andB(ui(x, t, ω))|x∈∂D = E [h(x, t, ω)Yi(ω, t)]. If the periodic boundary condition is used in Eq.(1c), u and thespatial basis U satisfy the periodic boundary condition.
Remark 2.1. In the event that two eigenvalues approach each other and then separate as time increases,we refer to this as eigenvalue crossings. The breakdown in determining matrices C and D when we haveeigenvalue crossings represents the extra degrees of freedom in choosing a complete set of orthogonal eigen-vectors for repeated eigenvalues. To overcome this difficulty associated with eigenvalue crossings, we candetect such an event and temporarily freeze the spatial basis U or the stochastic basis Y and continue toevolve the system for a short duration. Once the two eigenvalues separate, the solution can be recast intothe bi-orthogonal form via KL expansion. Recall that one of U and Y is always kept orthogonal even duringthe “freezing” stage. We can devise a KL expansion algorithm which avoids the need to form the covariancefunction explicitly. More details will be discussed in the section about numerical implementation.
3. Theoretical analysis
3.1. Bi-Orthogonality Preservation
In the previous section, we have used the bi-orthogonal condition in deriving our DyBO method. Sincewe have made a number of approximations by using a finite truncation of the KL expansion and performingvarious projections, these formal derivations do not necessarily guarantee that the DyBO method actuallypreserves the bi-orthogonal condition. In this section, we will show that the DyBO formulation (16) preservesthe bi-orthogonality of the spatial basis U and the stochastic basis Y for t ∈ [0, T ] if the bi-orthogonalitycondition is satisfied initially.
Theorem 3.1 (Preservation of Bi-Orthogonality in DyBO formulation). Assume that there is no eigenvaluecrossing up to time T > 0, i.e. ‖uj‖L2(D) 6= ‖ui‖L2(D) for i 6= j. The solutions U and Y of system
(16) satisfy the bi-orthogonality condition (5) exactly as long as the initial conditions U(x, 0) and Y(ω, 0)satisfy the bi-orthogonality condition (5). Moreover, if we write χ(i,j)(t) = 〈ui, uj〉 (t) for i > j and denote
χ(t) =(χ(i,j)(t)
)i>j
as a column vector in Rm(m−1)
2 ×1, then there exists an anti-symmetric matrix W(t) ∈
Rm(m−1)
2 ×m(m−1)2 such that
dχ(t)
dt=W(t)χ(t), (17a)
χ(0) = 0. (17b)
Similar results also hold for the stochastic basis Y.
Remark 3.1. The fact that W(t) is anti-symmetric is important for the stability of our DyBO method inpreserving the dynamic bi-orthogonality of the spatial and stochastic basis. Due to numerical round-offerrors, discretization errors, etc, U and Y may not be perfectly bi-orthogonal at the beginning or becomeso later in the computation. The above theorem sheds some lights on the numerical stability of DyBOformulation in this scenario. Since the eigenvalues of an anti-symmetric matrix are purely imaginary or zeroand their geometric multiplicities are always one (see page 115, [22]), so any deviation from the bi-orthogonalform will not be amplified.
Proof of Theorem 3.1. We only give the proof for the spatial basis U, since the proof for the stochastic basisY is similar. Direct computations give
d
dt
⟨UT , U
⟩=
⟨∂UT
∂t, U
⟩+
⟨UT ,
∂U
∂t
⟩= −D
⟨UT , U
⟩+⟨E[LuYT
], U⟩−⟨UT , U
⟩DT +
⟨UT , E
[LuY
]⟩= −D
⟨UT , U
⟩−⟨UT , U
⟩DT +GT∗ΛU + ΛUG∗, (18)
8
where we have used the definition of G∗ (see Theorem 2.1 and (A.13)) in the last equality. For each entrywith i 6= j in (18), we get,
d
dt〈ui, uj〉 = −
m∑k=1
Dik 〈uk, uj〉 −m∑l=1
〈ui, ul〉Djl +G∗ji ‖uj‖2L2(D) +G∗ij ‖ui‖2L2(D)
= −m∑
k=1,k 6=j
Dik 〈uk, uj〉 −m∑
l=1,l 6=i
〈ui, ul〉Djl
−Dij ‖uj‖2L2(D) −Dji ‖ui‖2L2(D) +(‖uj‖2L2(D) − ‖ui‖
2L2(D)
)Dij
= −m∑
k=1,k 6=i,j
Dik 〈uk, uj〉 −m∑
l=1,l 6=i,j
Djl 〈ui, ul〉 , (19)
where Eq. (15d) and the anti-symmetric property of matrix D are used. From Eq. (19), we see that χ(t)satisfies a linear ODE system in the form of (17) with matrix W(t) given in terms of Dij and the initialcondition comes from the fact that U(x, 0) are a set of orthogonal basis. It is easy to see that, as long asthe solutions of system (16) exist, W(t) is well-defined and the ODE system admits only zero solution, i.e.,orthogonality of U is preserved for t ∈ [0, T ]. Similar arguments can be used to show Y remains orthonormalfor t ∈ [0, T ].
Next, we show that the matrix W in ODE system (17) is anti-symmetric. Consider two pairs of indices(i, j) and (p, q) with i > j and p > q. The corresponding equations for χ(i,j) and χ(p,q) are
dχ(i,j)
dt= −
m∑k=1,k 6=i,j
Dikχ(k,j) −m∑
l=1,l 6=i,j
Djlχ(i,l), (20a)
dχ(p,q)
dt= −
m∑k=1,k 6=p,q
Dpkχ(k,q) −m∑
l=1,l 6=p,q
Dqlχ(p,l), (20b)
where we have identified χ(i,j) with χ(j,i). First, we consider the case two index pairs are identical. We seefrom Eq. (20a) that there is no χ(i,j) on the right side, so W(i,j),(i,j) = 0. Next, we consider the case noneof indices in one index pair is equal to any in the other, i.e., i 6= p, q and j 6= p, q. It is obvious that noχ(p,q) appears in the summations on the right side of Eq. (20a), which implies W(i,j),(p,q) = 0. Similarly,we have W(p,q),(i,j) = 0. Last, we consider the case in which one index in one index pair is equal to oneindex in another index pair but the remaining two indices from two index pair are not equal. Without lossof generality, we assume i = p and j 6= q. From Eq. (20a), we have
dχ(i,j)
dt= · · · −Djqχ(i,q) − · · · = · · · −Djqχ(p,q) − · · · ,
where we have intentionally isolated the relevant term from the second summation and the last equality isdue to i = p. Similarly, from Eq. (20b), we have
dχ(p,q)
dt= · · · −Dqjχ(p,j) − · · · = · · · −Dqjχ(i,j) − · · · .
The above two equations imply that W(i,j),(p,q) = −W(p,q),(i,j). Similar arguments apply to other cases.This completes the proof.
3.2. Error Analysis
In this section, we will consider the convergence of the solution obtained by the DyBO formulation (16)to the original SPDE (1). Theorem 3.2 gives the error analysis of the DyBO formulation.
9
Theorem 3.2 (The consistency between the DyBO formulation and the original SPDE). The stochasticsolution of system (16) satisfies the following modified SPDE
∂u
∂t= Lu+ em, (21)
where em is the error due to the m-term truncation and
em = −ΠY(ΠU(Lu)). (22)
Proof. We show consistency by computing directly from Eqs. (16b)and (16c) or equivalently Eqs. (??)and (??),
UdYT
dt= Lu+ UDTYT −
(Lu−UΛ−1
U
⟨UT , Lu
⟩)−UΛ−1
U
⟨UT , E
[LuY
]⟩YT ,
∂U
∂tYT = UCYT + YE
[LuYT
]−Y
⟨E[LuYT
], U⟩
Λ−1U UT .
Combining the above two, we have
UdYT
dt+∂U
∂tYT = Lu+ em,
where
em = −(Lu−UΛ−1
U
⟨UT , Lu
⟩)+ U
(DT + C
)YT − UΛ−1
U
⟨UT , E
[LuY
]⟩YT
+ YE[LuYT
]−Y
⟨E[LuYT
], U⟩
Λ−1U UT
= −(Lu−UΛ−1
U
⟨UT , Lu
⟩)+ E
[(Lu−UΛ−1
U
⟨UT , Lu
⟩)Y]
YT ,
where Eq. (14c) has been used in deriving the first equality. By noticing Lu = Lu − E [Lu], we proveEq. (21).
Remark 3.2. According to Theorem 3.1, the bi-orthogonality of U and Y are preserved for all time. Becauseboth Hilbert space L2 (D) and L2 (Ω) are separable, the spatial basis U and the stochastic basis Y becomea complete basis for L2 (D) and L2 (Ω) as m→ +∞, respectively, which implies limm→+∞ em = 0.
Next we consider a special case where the differential operator L is deterministic and linear, such as
Lv = c(x) ∂v∂x , or Lv = − ∂∂xi
(aij(x) ∂v∂xj
), where c(x) and (aij(x)) are deterministic functions. Only initial
conditions are assumed to be random, i.e., the randomness propagates into the system only through initialconditions.
Corollary 3.3. Let the differential operator L be deterministic and linear. The residual in Eq. (21) is zerofor all time, i.e.,
em(t) ≡ 0, t > 0.
Proof. This can be seen by directly computing the truncation error em. First we substitute Lu = Lu−E [Lu]into Eq. (22),
em = −ΠY
(Lu− E [Lu]−UΛ−1
U
⟨UT , Lu− E [Lu]
⟩).
Since the differential operator L is deterministic and linear, we have Lu = Lu + LUYT , where LU =(Lu1,Lu2, · · · ,Lum). This gives
em = −ΠY
(Lu+ LUYT − E
[Lu+ LUYT
]−UΛ−1
U
⟨UT , Lu− E [Lu]
⟩)= −ΠY
(LUYT −UΛ−1
U
⟨UT , LU
⟩YT)
= −ΠY
(Y(LUT −
⟨LUT , U
⟩Λ−1
U UT))
= 0.
The last line is due to the orthogonal complementary project.
10
The above corollary implies that DyBO is exact if the randomness can be expressed in a finite-term KLexpansion. The next corollary concerns a slightly different case where the differential operator L is affine inthe sense that Lu = Lu+f(x, t, ω) and the differential operator L is linear and deterministic. The stochasticforce f(·, t, ·) ∈ L2 (D × Ω) for all t.
Corollary 3.4. If the differential operator L is affine, i.e., Lu = Lu + f(x, t, ω), and f is a second-orderstochastic process at each fixed time t, the residual in Eq. (22) is given below
em = −ΠYΠU(f).
Proof. Again by directly computing, we have by the linearity of the differential operator L
Lu = Lu+ LUYT + f.
Simple calculation shows that
Lu = Lu− E [Lu] = LUYT + f − E [f ] .
Substituting into eqn. (22), we complete the proof.
Remark 3.3. For this special case, Corollary 3.4 implies that numerical solutions are accurate as long asthe spatial basis U and the stochastic basis Y provide good approximations to the external forcing term f ,which is not surprising at all.
4. Numerical Implementation
4.1. Representation of Stochastic Basis Y
The DyBO formulation (16) is a combination of deterministic PDEs (16a) and (16b) and SODEs (16c).For the deterministic PDEs in the DyBO formulation, we can apply suitable spatial discretization schemesand numerical integrators to solve them numerically. One of the conceptual difficulties, from the viewpoint ofthe classical numerical PDEs, involves representations of random variables or functions defined on abstractprobability space Ω. Essentially, there are three ways to represent numerically stochastic basis Y(ω, t):ensemble representations in sampling methods, such as Monte carlo method, sparse grid based stochasticcollocation method [1, 8] and spectral representations, such as the generalized Polynomial Chaos method[54, 55, 51, 28, 34] or Wavelet based Chaos method [36].
4.1.1. Ensemble Representation (DyBO-MC)
In the MC method framework, the stochastic basis Y(ω, t) are represented by an ensemble of realizations,i.e.,
Y(ω, t) ≈ Y(ω1, t),Y(ω2, t),Y(ω3, t), · · · ,Y(ωNr, t) ,
where Nr is the total number of realizations. Then the expectations in the DyBO formulation (16) can bereplaced by ensemble averages, i.e.,
∂u
∂t(x, t) =
1
Nr
Nr∑i=1
Lu(x, t, ωi), (23a)
∂U
∂t(x, t) = −U(x, t)D(t)T +
1
Nr
Nr∑i=1
Lu(x, t, ωi)Y(ωi, t), (23b)
dY
dt(ωi, t) = −Y(ωi, t)C(t)T +
⟨Lu(x, t, ωi), U(x, t)
⟩ΛU(t)−1, (23c)
11
where C(t) and D(t) can be solved from (15) with
G∗(u,U,Y) = Λ−1U
⟨UT ,
1
Nr
Nr∑i=1
Lu(x, t, ωi)Y(ωi, t)
⟩, (24)
and
Lu(x, t, ωi) = Lu(x, t, ωi)−1
Nr
Nr∑i=1
Lu(x, t, ωi). (25)
The above system is the MC version of DyBO method and we denote it as DyBO-MC. Close examinationsreveal that Eq. (23a) for expectation and Eq. (23b) for spatial basis are only solved once for all the realizationsat each time iteration while Eq. (23c) for the stochastic basis is decoupled from realization to realizationand can be solved simultaneously across all realizations. In this sense, we see that DyBO-MC is semi-sampling and it preserves some desirable features of MC method, for example, the convergence rate doesnot depend on stochastic dimensionality and the solution process of Y (ωi, t)’s are decoupled. Clearly, not
only the number of realizations, but also how these realizations ωiNr
i=1 are chosen in Ω affects the numericalaccuracy and the efficiency. Quasi Monte Carlo method [43, 41], variance reduction and other techniquesmay be combined into DyBO-MC to further improve its efficiency and accuracy. However, the disadvantageof MC method is its slow convergence. Developing effective sampling version of DyBO method such as themulti-level Monte Carlo method [24] is currently under investigation.
4.1.2. Spectral Representation (DyBO-gPC)
In many physical and engineering problems, randomness generally comes from various independen-t sources, so randomness in SPDE (1) is often given in terms of independent random variables ξi(ω).Throughout this paper, we assume only a finite number of independent random variables are involved, i.e.,ξ(ω) = (ξ1(ω), ξ2(ω), · · · , ξr(ω)), where r is the number of such random variables. Without loss of gen-erality, we can further assume they all have identical distribution ρ(·). Thus, the solution of SPDE (1)is a functional of these random variables, i.e., u(x, t, ω) = u(x, t, ξ(ω)). When SPDE is driven by someknown stochastic process, such as Brownian Motion Bt, by Cameron-Martin theorem [9], the stochasticsolution can be approximated by a functional of identical independent standard Gaussian variables, i.e.,u(x, t, ω) ≈ u(x, t, ξ(ω)) with ξi being a standard normal random variable.
In this paper, we only consider the case where randomness is given in terms of independent standardGaussian random variables, i.e., ξi ∼ N (0, 1) and H are a set of tensor products of Hermite polynomials.In this case, the method described above is also known as the Wiener-Chaos Expansion [52, 9, 12, 13, 28].When the distribution of ξi is not Gaussian, the discussions remain almost identical and the major differenceis the set of orthogonal polynomials.
We denote by Hi(ξ)∞i=1 the one-dimensional polynomials orthogonal to each other with respect to thecommon distribution ρ(z), i.e., ∫ ∞
−∞Hi(ξ)Hj(ξ)ρ(ξ) dξ = δij .
For some common distributions, such as Gaussian or uniform distribution, such polynomial sets are well-known and well-studied, many of which fall in the Ashley scheme, see [54]. For general distributions,such polynomial sets can be obtained by numerical methods, see [51]. Furthermore, by a tensor productrepresentation, we can use the one-dimensional polynomial Hi(ξ) to construct a complete orthonormal basisHα(ξ)’s of L2 (Ω) as follows
Hα(ξ) =
r∏i=1
Hαi(ξi), α ∈ J∞r ,
where α is a multi-index, i.e., a row vector of non-negative integers,
α = (α1, α2, · · · , αr) αi ∈ N, αi ≥ 0, i = 1, 2, · · · , r,
12
and J∞r is a multi-index set of countable cardinality,
J∞r = α = (α1, α2, · · · , αr) |αi ≥ 0, αi ∈ N \ 0 .
We have intentionally removed the zero multi-index corresponding to H0(ξ) = 1 since it is associated withthe mean of the stochastic solution and it is better to deal with it separately according to our experience.When no ambiguity arises, we simply write the multi-index set J∞r as J∞. Clearly, the cardinality of J∞ris infinite. For the purpose of numerical computations, we prefer a polynomial set of finite size. There aremany choices of truncations, such as the set of polynomials whose total orders are at most p, i.e.,
Jpr =
α |α = (α1, α2, · · · , αr) , αi ≥ 0, αi ∈ N, |α| =
r∑i=1
αi ≤ p
\ 0 ,
and sparse truncations proposed in [28], see also Luo’s thesis [33]. Again, we may simply write such atruncated set as J when no ambiguity arises. The cardinality of J, or the total number of polynomial basis
functions, denoted by Np = |J|, is equal to (p+r)!p!r! . As we can see, Np grows exponentially fast as p and r
increase. This is also known as the curse of dimensionality.By the Cameron-Martin theorem [9], we know that the solution of SPDE (1) admits a generalized
Polynomial Chaos expansion (gPCE)
u(x, t, ω) = v(x, t) +∑
α∈J∞vα(x, t)Hα(ξ(ω)) ≈ v(x, t) +
∑α∈J
vα(x, t)Hα(ξ(ω)). (26)
If we write
HJ(ξ) =(Hα1
(ξ),Hα2(ξ), · · · ,HαNP
(ξ))αi∈J
,
V(x, t) =(vα1
(x, t), vα2(x, t), · · · , vαNp
(x, t))αi∈J
,
both of which are row vectors, the above gPCE can be compactly written in a vector form
ugPC(x, t, ω) = v(x, t, ω) = v(x, t) + V(x, t)H(ξ)T . (27)
By substituting the above gPCE into Eq. (1a), it is easy to derive the gPC formulation for SPDE (1),
∂v
∂t= E [Lv] , (28a)
∂V
∂t= E
[LvH
]. (28b)
Now, we turn our attention to the gPC version of DyBO formulation (DyBO-gPC). The Cameron-Martintheorem also implies the stochastic basis Yi(ω, t)’s in the KL expansion (7) can be approximated by the linearcombination of polynomials chaos, i.e.,
Yi(ω, t) =∑α∈J
Hα(ξ(ω))Aαi(t), i = 1, 2, · · · ,m, (29)
or in a matrix form,Y(ω, t) = H (ξ(ω)) A, (30)
where A ∈ RNp×m. The expansion (7) now reads
u = u+ UATHT .
13
We can derive equations for u, U and A, instead of u, U and Y. In other words, the stochastic basis Yare identified with a matrix A ∈ RNp×m in DyBO-gPC. Substituting Eq. (30) into Eq. (16c) and using theidentity E
[HTH
]= I, we get the DyBO-gPC formulation of SPDE (1),
∂u
∂t= E [Lu] , (31a)
∂U
∂t= −UDT + E
[LuH
]A, (31b)
dA
dt= −ACT +
⟨E[HT Lu
], U⟩
Λ−1U , (31c)
where C(t) and D(t) can be solved from (15) with
G∗(u,U,Y) = Λ−1U
⟨UT , E
[LuY
]⟩= Λ−1
U
⟨UT , E
[LuH
]⟩A. (32)
By solving the system (31), we have an approximate solution to SPDE (1)
uDyBO-gPC = u+ UATHT ,
or simply uDyBO when no ambiguity arises.
4.1.3. gSC Representation (DyBO-gSC)
In our DyBO-gPC method, the stochastic process Yi(ξ(ω), t) is projected on to the gPC basis H andreplaced by gPC coefficients, i.e., Aαi, which can be computed by
Aαi = E [Yi(ω)Hα(ξ(ω))] .
Numerically, the above integral can be evaluated with high accuracy on some sparse grid. In other words,the stochastic basis Y can also be represented as an ensemble of realizations Y(ωi) where ξ(ωi)’s are nodesfrom some sparse grid and associated with certain weight wi. The gSC version of DyBO formulation, whichwe denote as DyBO-gSC, is
∂u
∂t(x, t) =
Ns∑i=1
wiLu(x, t, ωi),
∂U
∂t(x, t) = −U(x, t)D(t)T +
Ns∑i=1
wiLu(x, t, ωi)Y(ωi, t),
dY
dt(ωi, t) = −Y(ωi, t)C(t)T +
⟨Lu(x, t, ωi), U(x, t)
⟩ΛU(t)−1, i = 1, 2, ..., Ns,
where ωiNs
i=1 are the sparse grid points, wiNs
i=1 are the weight, Ns is the number of sparse grid pointsand C(t), D(t) can be solved from (15) with
G∗(u,U,Y) = Λ−1U
⟨UT ,
Ns∑i=1
wiLu(x, t, ωi)Y(ωi, t)
⟩. (34)
The primary focus of this paper is on DyBO-gPC methods. We will provide details and numerical examplesof DyBO-gSC methods in our future work.
4.2. Eigenvalue Crossings
As the system evolves, eigenvalues of different basis in the KL expansion of the SPDE solution mayincrease or decrease. Some of them may approach each other at some time, cross and then separate as
14
Time t
Eigenvalueλi
λ1
λ2
λ3
t∗1 t∗2
Y-Stage
U-Stage
U(x, t) ≡ U(x, s)
Y(ω, t) ≡ Y(ω, s)
τ < δoutτ
Enterin
g:τ<δinτ
Exitin
g:τ>δout
τ
s+ kY dt s+ 3kY dt
s
s
Figure 2: Illustration of Eigenvalue Crossing
illustrated in Fig. 2. In the figure, λ1 and λ2 cross each other at t∗1 and λ1 and λ3 cross each other at t∗2. Inthis case, if C and D continue to be solved from the linear system (14) via (15), numerical errors will pollutethe results. Here, we propose to freeze U or Y temporarily for a short time and continue to evolve thesystem using different equations as derived below. At the end of this short duration, the solution is recastinto the bi-orthogonal form via the KL expansion, which can be achieved efficiently since U or Y is stillkept orthogonal in this short duration. We are also currently exploring alternative approach to overcomethe eigenvalue crossing problem, and will report our result in a subsequent paper.
In order to apply the above strategy, we have to be able to detect that two eigenvalues may potentiallycross each other in the near future. There are several ways to detect such crossing. Here we propose tomonitor the following quantity
τ = mini 6=j
|λi − λj |max(λi, λj)
. (35)
Once this quantity drops below certain threshold δinτ ∈ (0, 1), we adopt the algorithm of freezing U or Y to
evolve the system and continue to monitor this quantity τ . When τ exceeds some pre-specified thresholdδoutτ ∈ (0, 1), the algorithm recasts the solution in the bi-orthogonal form via an efficient algorithm detailed
in the next two sub-sections and continues to evolve the DyBO system. Threshold δinτ and δout
τ may beproblem-dependent. In our numerical experiments, we found δin
τ = 1% and δoutτ = 1% gave accurate results.
Here we only describe the basic idea of freezing stochastic basis Y or Y-Stage algorithm. The methodof freezing the spatial basis U or U -Stage algorithm can be obtained analogously, see [10] for more details.Suppose at some time t = s, potential eigenvalue crossing is detected and the stochastic basis Y are frozenfor a short duration ∆s, i.e., Y(ω, t) ≡ Y(ω, s) for t ∈ [s, s + ∆s], which we call Y-stage. Because Y areorthonormal, the solution of SPDE (1) admits the following approximation of a truncated expansion.
u(x, t, ω) = u(x, t) +
m∑i=1
ui(x, t)Yi(ω, s) = u(x, t) + U(x, t)Y(ω, s)T . (36)
It is easy to derive a new evolution system for this stage,
∂u
∂t= E [Lu] , (37a)
∂U
∂t= E
[LuY
]. (37b)
During this stage, Y is unchanged and orthogonal, but U changes in time and would not maintain itsorthogonality. The solution is no longer bi-orthogonal, so eigenvalues cannot be computed via λi = ‖ui‖2L2(D)
15
any more. Next, we derive formula for eigenvalues in this stage and then show how the solutions are recastinto the bi-orthogonal form at the end of the stage, i.e., t = s+ ∆s.
The covariance function can be computed as
Covu(x, y) = E [(u(x)− u(x)) (u(y)− u(y))] = E[U(x)YTYUT (y)
]= U(x)UT (y),
where t is omitted for simplicity and E[YTY
]= I is used. By some stable orthogonalization procedures,
such as the modified Gram-Schmidt algorithm, U(x) can be written as
U(x) = Q(x)R, (38)
where Q(x) = (q1(x), q2(x), · · · , qm(x)), qi(x) ∈ L2 (D) for i = 1, 2, · · · ,m,⟨QT (x), Q(x)
⟩= I and R ∈
Rm×m. Here RRT ∈ Rm×m is a positive definite symmetric matrix and its SVD decomposition reads
RRT = WΛRWT , (39)
where W ∈ Rm×m is an orthonormal matrix, i.e., WWT = WTW = I, and ΛR is a diagonal matrix withpositive diagonal entries. The computational cost of W and ΛR is negligible compared to other parts of thealgorithm. The covariance function can be rewritten as
Covu(x, y) = Q(x)RRTQT (y) = Q(x)WΛRWTQT (y).
Now, it is easy to see that the eigenfunctions of covariance function Covu(x, y) are
U(x) = Q(x)WΛ12
R, (40)
and eigenvalues are diag (ΛR) because⟨UT , U
⟩=⟨Λ
12
RWTQT , QWΛ12
R
⟩= ΛR,
where we have used the orthogonality of Q and W. To compute Y, we start with the identity
YUT = YUT ,
and multiply row vector U on both sides from the right and take inner product 〈·, ·〉,
Y⟨UT , U
⟩= Y
⟨UT , U
⟩,
where ⟨UT , U
⟩= ΛR,⟨
UT , U⟩
=⟨RTQT (x), Q(x)WΛ
12
R
⟩= RTWΛ
12
R.
So we obtain the stochastic basis Y, which is given by
Y = YRTWΛ− 1
2
R . (41)
If a generalized polynomial basis H is adopted to represent Y, we freeze A instead, i.e., A(t) ≡ A(s) andthe system (37) is replaced by
∂u
∂t= E [Lu] , (42a)
∂U
∂t= E
[LuH
]A, (42b)
where A(t) ≡ A(s). At the exiting point, Eq. (41) is replaced by
A = ARTWΛ− 1
2
R . (43)
As we can see, the computation of eigenvalues in this stage is not trivial, so we do not want to computeτ every time iteration. Instead, τ is only evaluated every kY time steps to achieve a balance betweencomputational efficiency and accuracy. See the zoom-in figure in Fig. 2 for illustration.
16
5. Numerical Examples
Previous sections highlight the analytical aspects of the DyBO formulation and its numerical algorithm.In this section, we demonstrate the effectiveness of this method by several numerical examples with increasinglevel of difficulties, each of which emphasizes and verifies some of analytical results in the previous sections.In the first example, we consider a SPDE which is driven purely by stochastic forces. The purpose of thisexample is to show that the DyBO formulation tracks the KL expansion. Corollary 3.3 regarding the errorpropagation of the DyBO method is numerically verified in the second numerical example where a transportequation with a deterministic velocity and random initial conditions is considered. In the last example,we consider Burgers’ equation driven by a stochastic force as an example of a nonlinear PDE driven bystochastic forces. We demonstrate the convergence of the DyBO method with respect to the number ofbasis pairs, m. In the second part of this paper [11], we will consider the more challenging Navier-Stokesequations and the Boussinesq approximation with random forcings. Some important numerical issues, suchas adaptivity, parallelization and computational complexity, will be also studied in details.
5.1. SPDE Purely Driven by Stochastic Force
To study the convergence of our DyBO-gPC algorithms, it is desirable to construct a SPDE whosesolution and KL expansion are known analytically. This would allow us to perform a thorough convergencestudy for our algorithm especially when we encounter eigenvalue crossings. In this example, we considera SPDE which is purely driven by a stochastic force f , i.e., ∂u
∂t = Lu = f(x, t, ω). The solution can be
obtained by direct integration, i.e., u(x, t, ω) = u(x, 0, ω) +∫ t
0f(x, s, ω) ds. Specifically, in this section, we
consider the SPDE∂u
∂t= Lu = f(x, t, ξ(ω)), x ∈ D = [0, 1], t ∈ [0, T ], (44)
where ξ = (ξ1, ξ2, · · · , ξr) are independent standard Gaussian random variables, i.e., ξi ∼ N (0, 1), i =1, 2, · · · , r. The exact solution is constructed as follows
u(x, t, ξ) = v(x, t) + V(x, t)ZT (ξ, t), (45)
where V(x, t) = V(x)WV(t)Λ12
V(t), Z(ξ, t) = Z(ξ)WZ(t), V(x) = (v1(x), · · · , vm(x)) with 〈vi(x), vj(x)〉 =
δij and Z(ξ) =(Z1(ξ), · · · , Zm(ξ)
)with E
[ZiZj
]= δij for i, j = 1, 2, · · · ,m. WV(t) and WZ(t) are
m-by-m orthonormal matrices, and Λ12
V(t) is a diagonal matrix.Clearly, the exact solution u is intentionally given in the form of its finite-term KL expansion and the
diagonal entries of matrix ΛV(t) are its eigenvalues. By carefully choosing the eigenvalues, we can mimicvarious difficulties which may arise in more involved situations and devise corresponding strategies. Thestochastic forcing term on the right hand side of Eq. (44) can be obtained by differentiating the exactsolution, i.e.,
Lu = f =∂v
∂t+∂V
∂tZT + V
dZT
dt. (46)
The initial condition of SPDE (44) can be obtained by simply setting t = 0 in Eq. (45). From the generalDyBO-gPC formulation (16), simple calculations give the DyBO-gPC formulation for SPDE (44)
∂u
∂t=∂v
∂t, (47a)
∂U
∂t= −UDT +
(∂V
∂tWT
Z + VdWT
Z
dt
)E[ZTH
]A, (47b)
dA
dt= −ACT + E
[HT Z
]⟨WZ
∂VT
∂t+
dWZ
dtVT , U
⟩Λ−1
U , (47c)
17
and
G∗ (u,U,A) = Λ−1U
⟨U, ∂V
∂tWT
Z + VdWT
Z
dt
⟩E[ZTH
]A.
The initial conditions are simply
u(x, 0) = v(x, 0), U(x, 0) = V(x, 0), A(0) = E[HTZ(ξ, 0)
]= E
[HT Z
]WZ(0).
In the event of eigenvalue crossings, we have the Y-stage system from (42),
∂u
∂t=∂v
∂t, (48a)
∂U
∂t=
(∂V
∂tWT
Z + VdWT
Z
dt
)E[ZTH
]A, (48b)
or the U-stage system,
∂u
∂t=∂v
∂t, (49a)
dA
dt= E
[HT Z
]⟨WZ
∂VT
∂t+
dWZ
dtVT , U
⟩Λ−1
U . (49b)
In the numerical examples presented in this section, we consider a small system m = 3 and use thefollowing settings,
V(x) =(√
2 sin(πx),√
2 sin(5πx),√
2 sin(9πx)), Z(x) = (H1(ξ1),H2(ξ1),H3(ξ1)) ,
WV(t) = PV
cos bVt − sin bVt 0sin bVt cos bVt 0
0 0 1
PTV, WZ(t) = PZ
cos bZt − sin bZt 0sin bZt cos bZt 0
0 0 1
PTZ,
where bV = 2.0, bZ = 2.0, PV and PZ are two orthonormal matrices generated randomly, and H (ξ) =(H1(ξ1),H2(ξ1), · · · ,H5(ξ1)). Eigenvalues in the KL expansion of an SPDE solution may increase or decreaseas time increases. Some of them may approach each other at some time, cross and then separate later. Whentwo eigenvalues are close or equal to each other, numerical instability may arise in solving matrices C and Dvia (15). In Section 4.2, we have proposed a U-Stage by freezing the spatial basis U or a Y-Stage by freezingthe stochastic basis Y temporarily to resolve this issue. Here, we demonstrate the success by incorporatingsuch strategies into the DyBO algorithm. To this end, we choose T = 1.2 and eigenvalues
Λ12
V = diag (sin 2πt+ 2, cos 2πt+ 1.5, 1.8) ,
where eigenvalues cross each other at t ≈ 0.0675, 0.2015, 0.5320, 0.7985, 0.9650, 1.0675, see Fig. 5.In this numerical example, the time step ∆t = 1.0× 10−3 and kU = 20 in the U-stage , i.e., the exiting
condition is checked every 20 time iterations when the system is in the U-stage. The mean E [u], the standarddeviation (STD)
√Var (u) and the three spatial basis in KLE at time t = T are shown in Fig. 3 and Fig. 4,
respectively. Clearly, the results given by DyBO almost perfectly match the exact ones with L2 relativeerrors of mean and STD below 10−10 since only the spatial and temporal discretizations contribute to thenumerical errors in this example. In Fig. 5, eigenvalues are plotted as functions of time and zoom-in figuresare given at t = 0.0675 and t = 0.7985 to show eigenvalue crossings and invoking of the U-stage algorithm.
We also check the deviation of the computed spatial and stochastic basis from the bi-orthogonality by
monitoring〈ui, uj〉
‖ui‖L2(D)‖uj‖L2(D)for the spatial basis and E [YiYj ] for the stochastic basis. The results are shown
in Fig. 6. We observe that the deviation of the spatial basis from orthogonality is very small (< 10−10)throughout the computation. Large deviations of the stochastic basis from the orthogonality only occurwhen eigenvalues cross each other since the DyBO algorithm is not designed to preserve the orthogonalityof Y during the U-stage and orthogonality is only restored once the algorithm exits from the U-stage. Wehave also applied the Y-stage to overcome eigenvalue-crossing issues in this numerical examples and theresults are very similar.
18
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−1
−0.8
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
0.8
1Mean
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
1
2
3
4
5
6STD
Exact
DyBO
Exact
DyBO
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−1
−0.8
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
0.8
1Mean
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
1
2
3
4
5
6STD
Exact
DyBO
Exact
DyBO
Figure 3: Mean and STD computed by DyBO at time t = 1.2.
0 0.2 0.4 0.6 0.8
−4
−2
0
2
4
0 0.2 0.4 0.6 0.8−3
−2
−1
0
1
2
3
0 0.2 0.4 0.6 0.8−2
0
2
4
6
Exact
DyBO
Exact
DyBO
Exact
DyBO
0 0.2 0.4 0.6 0.8
−4
−2
0
2
4
0 0.2 0.4 0.6 0.8−3
−2
−1
0
1
2
3
0 0.2 0.4 0.6 0.8−2
0
2
4
6
Exact
DyBO
Exact
DyBO
Exact
DyBO
0 0.2 0.4 0.6 0.8
−4
−2
0
2
4
0 0.2 0.4 0.6 0.8−3
−2
−1
0
1
2
3
0 0.2 0.4 0.6 0.8−2
0
2
4
6
Exact
DyBO
Exact
DyBO
Exact
DyBO
Figure 4: The spatial basis (u1(x, t), u2(x, t), u3(x, t) from the left to the right) in KL expansion of SPDEsolution computed by DyBO at time t = 1.2.
0 0.2 0.4 0.6 0.8 1 1.2
0
1
2
3
4
5
6
7
8
9
0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16
5.4
5.6
5.8
6
6.2
6.4
0.77 0.78 0.79 0.8 0.81 0.82 0.83 0.842.9
3
3.1
3.2
3.3
3.4
3.5
λ1
λ2
λ3
U−Stage U−Stage
Figure 5: Eigenvalues computed by DyBO. Two zoom-in figures are provided when eigenvalues cross.
0 0.2 0.4 0.6 0.8 1 1.2 1.4−2
0
2
4x 10
−11
0 0.2 0.4 0.6 0.8 1 1.2 1.40
2
4
6
8
10
λi
0 0.2 0.4 0.6 0.8 1 1.2 1.4−1
−0.5
0
0.5
1E(Y)
0 0.2 0.4 0.6 0.8 1 1.2
−0.05
0
0.05
0.1
0 0.2 0.4 0.6 0.8 1 1.2 1.40
0.5
1
1.5x 10
−11 ubar L2 Error
0 0.2 0.4 0.6 0.8 1 1.2 1.40
0.02
0.04
0.06
0.08U L2 Error
u1 & u
2
u2 & u
3
u3 & u
1
Y1 & Y
2
Y2 & Y
3
Y3 & Y
1
(a) Orthogonality of the spatial basis U(x, t).
0 0.2 0.4 0.6 0.8 1 1.2 1.4−2
0
2
4x 10
−11
0 0.2 0.4 0.6 0.8 1 1.2 1.40
2
4
6
8
10
λi
0 0.2 0.4 0.6 0.8 1 1.2 1.4−1
−0.5
0
0.5
1E(Y)
0 0.2 0.4 0.6 0.8 1 1.2
−0.05
0
0.05
0.1
0 0.2 0.4 0.6 0.8 1 1.2 1.40
0.5
1
1.5x 10
−11 ubar L2 Error
0 0.2 0.4 0.6 0.8 1 1.2 1.40
0.02
0.04
0.06
0.08U L2 Error
u1 & u
2
u2 & u
3
u3 & u
1
Y1 & Y
2
Y2 & Y
3
Y3 & Y
1
(b) Orthogonality of the stochastic basis Y(ω, t).
Figure 6: Bi-orthgonality of spatial and stochastic basis.
19
5.2. Linear Deterministic Differential Operators with Random Initial Conditions
In this section, we consider a linear PDE with random initial conditions, i.e., the differential operator Lis deterministic and linear. For simplicity, we assume periodic boundary conditions.
∂u
∂t= Lu, x ∈ [0, 1], t ∈ [0, 0.4], (50a)
u(0, t, ξ) = u(1, t, ξ), (50b)
u(x, 0, ξ) = u(x, ξ), (50c)
where ξ = (ξ1, ξ2, · · · , ξr) are independent standard Gaussian random variables.Consider the gPC expansion of solution u = v + VHT . From Eq. (28), it is easy to obtain the gPC
formulation of this SPDE,
∂v
∂t= Lv, (51a)
∂V
∂t= LV, (51b)
where the initial conditions v(x, 0) = E [u] and V(x, 0) = E [u(x)H]. Clearly, the gPC formulation (51) islinear and provides the exact solution to the original SPDE (50) if the finite-term gPC expansion of theinitial condition is exact, i.e., u = E [u] + E [uH] HT .
Now consider the KL expansion of the solution, u = u + UYT = u + UATHT . From the generalDyBO-gPC formulation (16), simple calculations give the DyBO-gPC formulation for the SPDE (50)
∂u
∂t= Lu, (52a)
∂U
∂t= −UDT + LU, (52b)
dA
dt= −ACT + A
⟨LUT , U
⟩Λ−1U , (52c)
where C and D can be computed via Eq. (15) from G∗ = Λ−1U
⟨UT , LU
⟩.
If we compare the DyBO formulation (52) and the gPC formulation (51) for the linear SPDE (50),we observe the following interesting phenomenon. Although the original SPDE is linear and the gPCformulation remains linear, the DyBO formulation is clearly not linear. However, this is not surprisingbecause KL expansion is not a linear procedure. On the other hand, the gPC expansion is indeed a linearprocedure. As we argue in Sec. 3.2, our DyBO formulation essentially tracks the KL expansion of the exactsolution. Therefore, the non-linearity of KLE must be naturally built into the DyBO formulation to enablesuch tracking.
Corollary 3.3 implies that the solution given by the DyBO formulation is exact if the initial condition ucan be expressed exactly by a finite-term KL expansion. To verify this numerically, we consider a transportequation, i.e., L = − sin(2π(x+ 2t)) ∂
∂x and the initial condition is a functional of three standard Gaussianrandom variables, i.e., r = 3,
u(x, ξ) = cos(2πx) +
(sin(2πx),
1
2sin(4πx),
1
3sin(6πx)
)ATHT
J ,
where J =α |α ∈ J4
3, α3 ≤ 3\ 0. Matrix A is an orthonormal matrix generated randomly at the
beginning of our simulation.With time step ∆t = 1.0 × 10−3 and spatial grid size ∆x = 1/128, the elements in the spatial basis
computed by DyBO match perfectly the exact ones given by gPC as shown in Fig. 7. In Fig. 8, the L2
relative error of STD is plotted as a function of time t and remains very small (≤ 10−8). To confirm sucherrors are introduced mainly by the spatial and the temporal discretizations, we use another two sets of finergrid sizes, ∆t = 0.5× 10−3, ∆x = 1/256, and ∆t = 0.25× 10−3, ∆x = 1/512, and repeat the computations.The STD error drops significantly since we use spectral methods and a fourth order RK method, whichessentially verifies numerically Corollary 3.3.
20
0 0.2 0.4 0.6 0.8 1−1.5
−1
−0.5
0
0.5
1
1.5
0 0.2 0.4 0.6 0.8 1−1
−0.5
0
0.5
1
0 0.2 0.4 0.6 0.8 1−0.5
0
0.5
Exact
DyBo
Exact
DyBO
Exact
DyBO
0 0.2 0.4 0.6 0.8 1−1.5
−1
−0.5
0
0.5
1
1.5
0 0.2 0.4 0.6 0.8 1−1
−0.5
0
0.5
1
0 0.2 0.4 0.6 0.8 1−0.5
0
0.5
Exact
DyBo
Exact
DyBO
Exact
DyBO
0 0.2 0.4 0.6 0.8 1−1.5
−1
−0.5
0
0.5
1
1.5
0 0.2 0.4 0.6 0.8 1−1
−0.5
0
0.5
1
0 0.2 0.4 0.6 0.8 1−0.5
0
0.5
Exact
DyBo
Exact
DyBO
Exact
DyBO
Figure 7: The spatial basis (u1(x, t), u2(x, t), u3(x, t) from the left to the right) in the KL expansion of theSPDE solution computed by DyBO at time t = 0.4
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.40
1
2
3
4
5
6
7
x 10−9
δ t = 0.5e−3, nx=256
δ t =1.0e−3, nx=128
δ t = 0.25e−3, nx=512
Figure 8: L2 relative errors of STD computed by DyBO. The horizontal axis is time t.
5.3. Burgers’ equation driven by stochastic forces
In the next example, we consider the stochastic Burgers equation. Stochastic Burgers equation is wellknown for its rich structures due to the interaction between nonlinearity and randomness and their applica-tions in statistical mechanics, such as interfacial dynamics and directed polymers in random media and inthe context of turbulence. For more discussion regarding stochastic Burgers equation and its applications,please see [6, 18, 28, 33] and the reference therein.
Consider a one-dimensional Burgers equation driven by a zero-mean stochastic force f(x, t, ξ(ω)),
∂u
∂t= Lu = Lu+ f = −u∂u
∂x+ ν
∂2u
∂x2+ f, x ∈ [0, 1], t ∈ [0, T ], (53a)
u(x, 0, ξ) = u(x), (53b)
u(0, t, ξ) = u(1, t, ξ), (53c)
where the initial condition is deterministic and the boundary condition is periodic. Here, we call Lu =
−u∂u∂x + ν ∂2u∂x2 the deterministic Burgers’ differential operator or Burgers’ operator in short. To ensure the
stochastic solution does not blow up in a finite time, we assume that f(x, t, ξ) ∈ L2 (D × Ω) at any fixedtime t and the stochastic force admits a finite-term gPC expansion, i.e., f(x, t, ξ) = F(x, t)H(ξ)T , wherethe row vector F = (Fα)α∈J for some multi-index set J with |J| = Np.
When Burgers’ equation is driven by stochastic processes, such as the Brownian motion, the aboveformulation still provides a good approximate model. The exact form of the stochastic force f can beobtained by choosing certain orthonormal basis on L2 ([0, T ]) and projecting the Brownian path onto such
21
basis. See [28, 33] for details. We choose the stochastic force as f = σ(x) dBt
dt with σ(x) = 12 cos(4πx) and
dBtdt≈
r∑i=1
1√TMi
(t
T
)ξi =
r∑i=1
1√TMi
(t
T
)Hei
=
(1√TM1
(t
T
),
1√TM2
(t
T
), · · · , 1√
TMr
(t
T
), 0, 0, · · · , 0
)︸ ︷︷ ︸
Np terms
HT , (54)
where ei is a multi-index of length r whose ith entry is one and others are zeros, Mi(t)∞i=1 are a completeorthonormal basis of L2 ([0, 1]) and ξi’s are i.i.d. independent standard Gaussian random variables. Further,we use the following orthonormal basis for L2 ([0, T ]),
M1(t) = 1, Mi(t) =√
2 cos((i− 1)πt), i = 2, 3, · · · .
Consider the gPC expansion of the stochastic solution u = v + VHT . Simple calculations give the gPCformulation for the stochastic Burgers’ Eq.(53),
∂v
∂t= Lv −V
∂VT
∂x, (55a)
∂V
∂t=
(ν∂2V
∂x2− ∂ (vV)
∂x
)−(vα∂vβ∂x
T(H)αβγ
)1γ
+ F, (55b)
where T(H) is a third-order Np-by-Np-by-Np tensor, i.e., T(H)αβγ = E [HαHβHγ ] , α,β,γ ∈ J.
From the derivation of the gPC formulation, it is easy to see that the use of vector and tensor notationsnot only greatly reduces the complexity of the derivation, but also clearly reveals the structure of the gPCformulation and its relation to the deterministic Burgers’ equation. For example, the mean flow v is stilldriven by the deterministic Burgers’ differential operator L and exchanges energy with the stochastic flows
V through the term −V ∂VT
∂x . The stochastic flows V are convected by the mean flow through term −∂(vV)∂x ,
dissipate through term ν ∂2V∂x2 and interact with each others through −
(vα
∂vβ∂x T
(H)αβγ
)1γ
.
Next we consider the m-term truncated KL expansion of the solution u = u + UYT = u + UATHT .The DyBO-gPC formulation of the stochastic Burgers Eq. (53) is (see Appendix B for its derivation):
∂u
∂t= Lu−U
∂UT
∂x, (56a)
∂U
∂t= −UDT +
(ν∂2U
∂x2− ∂ (uU)
∂x
)−(ui∂uj∂x
AαiAβjAγkT(H)αβγ
)1k
+ FA, (56b)
dA
dt= A
(−CT +
⟨ν∂2UT
∂x2−∂(uUT
)∂x
, U
⟩Λ−1
U
)−(AαiAβjT
(U)ijk T
(H)αβγ
)γk
Λ−1U +
⟨FT , U
⟩Λ−1
U ,
(56c)
where matrices C and D can be solved from the linear system (14) with
ΛUG∗(u,U,Y) =
⟨UT , ν
∂2U
∂x2− ∂ (uU)
∂x
⟩−(T
(U)ijl AαiAβjAγkT
(H)αβγ
)lk
+⟨UT , F
⟩A.
In the following numerical examples, the classical fourth-order RK method is used as the ODE solver anda pseudo-spectral method is applied for the spatial discretizations. We choose r = 6 in the stochasticforce (54) and multi-index set J = J3
6 \ 0, which results in totally 83 terms in the gPC expansion, i.e.,|J| = 83. The spatial grid size ∆x is set to 1/128 and the time step ∆t is set to 0.5 × 10−3 for both gPCand DyBO, both of which are numerically integrated to time t = T = 1.0. 106 realizations are computed
22
in the MC method to approximate the exact solution with an error of order less than 10−3 according tothe Central Limit theorem. We choose the initial condition of the stochastic Burgers equation (53b) asu(x) = 1
2 (ecos(2πx) − 1.5) sin(2π(x+ 0.37)) and set the viscosity ν = 0.005.To understand the source of errors in the DyBO method, we decompose the error into two parts. The first
part of error is the difference between the DyBO solution u(DyBO,Jpr ,m) and the gPC solution u(gPC,Jp
r).Thesecond part of error is the difference between the exact solution u(Exact) and the gPC solution, u(gPC,Jp
r).More precisely, we have
E = u(DyBO,Jpr ,m) − u(Exact) =
u(DyBO,Jp
r ,m) − u(gPC,Jpr)
+u(gPC,Jp
r) − u(Exact)
= Em + EJ.
The first part of the error Em diminishes as m→∞, while the second part EJ is controlled by the multi-indexset J and goes to 0 as J→ J∞∞. Thus, there is no need to increase m any further once Em EJ. We writem∗ for such m and will verify the convergence of Em and calculate m∗ later.
Computational complexity of the DyBO system (56) involves considerably less work than that of thegPC system (55). Roughly speaking, for the gPC method, Np gPC coefficients, which are functions ofspatial variable x, are updated at each time iteration. However, for the DyBO method, only m spatialbasis functions and one Np-by-m matrix are updated at each time iteration. When the eigenvalues of thecovariance matrix decays rapidly which is true under certain assumptions [48], m is much smaller than Npand spatial grid number, leading to considerable computational saving.
Convergence to u(gPC,Jpr) - Error Em. In Fig. 9, we plot the L2 relative errors of mean and
STD computed by DyBO with m = 4, 6, 8, 10, 12 with respect to the gPC solution u(gPC,Jpr). Indeed, as
the number of mode pairs in DyBO increases, the L2 errors of mean and STD decreases, indicating theconvergence of u(DyBO,Jp
r ,m) to u(gPC,Jpr). The relative errors of mean and STD at time t = 1.0 are also
tabulated in the second and the third columns of Table. 1. With m = 12, both errors are below 0.3%. Weshould point out that the DyBO-adaptive is the adaptive algorithm of DyBO method which further reducesthe computational cost and improve the accuracy. The adaptive strategy will be discussed in details in thesecond part of our paper [11].
Direct Tracking of the KL Expansion. In Fig. 10, the first nine spatial basis at time t = T ,i.e., ui(x, T ), i = 1, 2, · · · , 9, are plotted and compared with the ones computed from u(gPC,Jp
r). Very goodagreements are observed for the first six eigenfunctions (the first two rows in Fig. 10). Although the 7th,8th and 9th spatial basis are under more influences of the unresolved mode pairs, i.e., m+ 1, m+ 2, etc. Westill observe good agreements for these spatial basis (the last row in Fig. 10). This is because the eigenvaluescorresponding to the 7th, 8th and 9th spatial basis are small.
In Fig. 11, we also plot the stochastic basis Y = HA. Since we never use directly the polynomial chaosset H in our computation, we plot in Fig. 11b the Np-by-m matrix A computed in our DyBO method andin Fig. 11c the one computed from the gPC solution , respectively. The stochastic basis are plotted for timet = T . The meaning of both figures may deserve some further explanations (see Fig. 11a). The verticalaxis from the top to the bottom is the multi-index α ∈ J, while the horizontal axis from the left to theright is the index of stochastic mode i = 1, 2, · · · ,m. In this setting, each column of such plot represents asingle column aj of matrix A, i.e., a stochastic mode Yj = Haj . Similarly, each row of such plot representsthe projections of all stochastic basis Y on a certain polynomial basis Hβ , i.e., E [YHβ ]. From Fig. 10 andFig. 11, we see that the solution of DyBO can effectively track the truncated KL expansion of the stochasticBurgers’ solution even if the differential operator is nonlinear.
Computational Speedup compared to gPC. Next we consider the efficiency of DyBO comparedto gPC. In the fourth and fifth columns of Table. 1, we also tabulate the relative errors of mean and STDcomputed by DyBO, with m = 4, 6, 8, 10, 12 and adaptive strategy, with respect to the “exact” solutionu(gPC,J∞r ) obtained by MC method with 106 realizations. The relative errors of mean and STD computedby the gPC method with respect to u(gPC,J∞r ) are also given in the first row. Clearly, when m = 10 orm = 12, the errors of DyBO are comparable to those of gPC listed in the fist row. These results show thatEm EJ when m = 10 or m = 12, further increasing m will not decrease the overall error E , i.e., m∗ = 10or m∗ = 12 in this numerical example. In this case, we achieve 9.6X ∼ 12.7X speedup.
23
0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.01
0.02
0.03
0.04
0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.01
0.02
0.03
0.04
0.05
0.06
0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
4
6
8
10
12
14
m=4
m=6
m=8
m=10
m=12
Adaptive
m=4
m=6
m=8
m=10
m=12
Adaptive
(a) Relative errors of mean0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.01
0.02
0.03
0.04
0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.01
0.02
0.03
0.04
0.05
0.06
0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
4
6
8
10
12
14
m=4
m=6
m=8
m=10
m=12
Adaptive
m=4
m=6
m=8
m=10
m=12
Adaptive
(b) Relative errors of STD
0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.01
0.02
0.03
0.04
0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.01
0.02
0.03
0.04
0.05
0.06
0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
4
6
8
10
12
14
m=4
m=6
m=8
m=10
m=12
Adaptive
m=4
m=6
m=8
m=10
m=12
Adaptive
(c) Number of mode pairs
Figure 9: L2 relative errors of mean and STD as functions of time.
Compared to u(gPC,Jpr) Compared to u(gPC,J∞r )
Methods Mean STD Mean STD Time (min)
gPC NA NA 1.1619% 2.1398% 42.1
DyBO m=4 3.219% 5.144% 3.096% 5.556% 1.32DyBO m=6 2.855% 4.522% 2.935% 4.512% 2.01DyBO m=8 0.930% 2.112% 1.333% 2.890% 2.64DyBO m=10 0.444% 0.983% 1.222% 2.375% 3.32DyBO m=12 0.171% 0.259% 1.109% 2.122% 4.37
Table 1: Relative errors of statistical quantities computed by DyBO and gPC at time t = 1.0.
Numerical confirmation of invariant measure. Finally we use the DyBO method to numericallyconfirm the invariance measure of stochastic Burgers equation. We choose r = 8 in the stochastic force (54)and multi-index set J = J4
8 \ 0. Sparse truncation [33] is used and results in totally 96 terms in gPCexpansion, i.e., |J| = 96. Spatial grid size ∆x is set to 1/128 and time step ∆t is set to 2.5 × 10−3 forboth gPC and DyBO, both of which are numerically integrated to time t = T = 6.0. We choose the initialcondition of the stochastic Burgers equation (53b) as u(x) = 0.5 cos(4πx), the spatial part of the stochasticforce as σ(x) = 0.5 cos(2πx) and set the viscosity ν = 0.005. It has been shown that the stochastic Burgers
equation with Brownian forcing (53) has solutions with invariant measure if∫ 1
0σ(x)dx = 0 (see e.g. [18]).
We plot the evolution history of the mean and STD of the solution. One can see that the mean and STDprofiles converge to a steady state. We should point out that the adaptive DyBO method has been used inthis example. Initially, we choose m = 4 elements in the spatial and stochastic basis, eventually we havem = 10. Once we reach the invariant measure, the number of modes remains the same.
24
0 0.2 0.4 0.6 0.8 1−0.4
−0.2
0
0.2
0.4
0 0.2 0.4 0.6 0.8 1−0.2
−0.1
0
0.1
0.2
0 0.2 0.4 0.6 0.8 1
−0.05
0
0.05
0 0.2 0.4 0.6 0.8
−0.04
−0.02
0
0.02
0.04
0.06
0.08
0 0.2 0.4 0.6 0.8 1−0.05
0
0.05
0 0.2 0.4 0.6 0.8 1−0.04
−0.02
0
0.02
0.04
0.06
0 0.2 0.4 0.6 0.8 1−0.06
−0.04
−0.02
0
0.02
0.04
0 0.2 0.4 0.6 0.8 1−0.04
−0.02
0
0.02
0.04
0.06
0 0.2 0.4 0.6 0.8 1−0.04
−0.02
0
0.02
0.04
0 0.2 0.4 0.6 0.8 1−0.4
−0.2
0
0.2
0.4
0 0.2 0.4 0.6 0.8 1−0.2
−0.1
0
0.1
0.2
0 0.2 0.4 0.6 0.8 1
−0.05
0
0.05
0 0.2 0.4 0.6 0.8
−0.04
−0.02
0
0.02
0.04
0.06
0.08
0 0.2 0.4 0.6 0.8 1−0.05
0
0.05
0 0.2 0.4 0.6 0.8 1−0.04
−0.02
0
0.02
0.04
0.06
0 0.2 0.4 0.6 0.8 1−0.06
−0.04
−0.02
0
0.02
0.04
0 0.2 0.4 0.6 0.8 1−0.04
−0.02
0
0.02
0.04
0.06
0 0.2 0.4 0.6 0.8 1−0.04
−0.02
0
0.02
0.04
0 0.2 0.4 0.6 0.8 1−0.4
−0.2
0
0.2
0.4
0 0.2 0.4 0.6 0.8 1−0.2
−0.1
0
0.1
0.2
0 0.2 0.4 0.6 0.8 1
−0.05
0
0.05
0 0.2 0.4 0.6 0.8
−0.04
−0.02
0
0.02
0.04
0.06
0.08
0 0.2 0.4 0.6 0.8 1−0.05
0
0.05
0 0.2 0.4 0.6 0.8 1−0.04
−0.02
0
0.02
0.04
0.06
0 0.2 0.4 0.6 0.8 1−0.06
−0.04
−0.02
0
0.02
0.04
0 0.2 0.4 0.6 0.8 1−0.04
−0.02
0
0.02
0.04
0.06
0 0.2 0.4 0.6 0.8 1−0.04
−0.02
0
0.02
0.04
0 0.2 0.4 0.6 0.8 1−0.4
−0.2
0
0.2
0.4
0 0.2 0.4 0.6 0.8 1−0.2
−0.1
0
0.1
0.2
0 0.2 0.4 0.6 0.8 1
−0.05
0
0.05
0 0.2 0.4 0.6 0.8
−0.04
−0.02
0
0.02
0.04
0.06
0.08
0 0.2 0.4 0.6 0.8 1−0.05
0
0.05
0 0.2 0.4 0.6 0.8 1−0.04
−0.02
0
0.02
0.04
0.06
0 0.2 0.4 0.6 0.8 1−0.06
−0.04
−0.02
0
0.02
0.04
0 0.2 0.4 0.6 0.8 1−0.04
−0.02
0
0.02
0.04
0.06
0 0.2 0.4 0.6 0.8 1−0.04
−0.02
0
0.02
0.04
0 0.2 0.4 0.6 0.8 1−0.4
−0.2
0
0.2
0.4
0 0.2 0.4 0.6 0.8 1−0.2
−0.1
0
0.1
0.2
0 0.2 0.4 0.6 0.8 1
−0.05
0
0.05
0 0.2 0.4 0.6 0.8
−0.04
−0.02
0
0.02
0.04
0.06
0.08
0 0.2 0.4 0.6 0.8 1−0.05
0
0.05
0 0.2 0.4 0.6 0.8 1−0.04
−0.02
0
0.02
0.04
0.06
0 0.2 0.4 0.6 0.8 1−0.06
−0.04
−0.02
0
0.02
0.04
0 0.2 0.4 0.6 0.8 1−0.04
−0.02
0
0.02
0.04
0.06
0 0.2 0.4 0.6 0.8 1−0.04
−0.02
0
0.02
0.04
0 0.2 0.4 0.6 0.8 1−0.4
−0.2
0
0.2
0.4
0 0.2 0.4 0.6 0.8 1−0.2
−0.1
0
0.1
0.2
0 0.2 0.4 0.6 0.8 1
−0.05
0
0.05
0 0.2 0.4 0.6 0.8
−0.04
−0.02
0
0.02
0.04
0.06
0.08
0 0.2 0.4 0.6 0.8 1−0.05
0
0.05
0 0.2 0.4 0.6 0.8 1−0.04
−0.02
0
0.02
0.04
0.06
0 0.2 0.4 0.6 0.8 1−0.06
−0.04
−0.02
0
0.02
0.04
0 0.2 0.4 0.6 0.8 1−0.04
−0.02
0
0.02
0.04
0.06
0 0.2 0.4 0.6 0.8 1−0.04
−0.02
0
0.02
0.04
0 0.2 0.4 0.6 0.8 1−0.4
−0.2
0
0.2
0.4
0 0.2 0.4 0.6 0.8 1−0.2
−0.1
0
0.1
0.2
0 0.2 0.4 0.6 0.8 1
−0.05
0
0.05
0 0.2 0.4 0.6 0.8
−0.04
−0.02
0
0.02
0.04
0.06
0.08
0 0.2 0.4 0.6 0.8 1−0.05
0
0.05
0 0.2 0.4 0.6 0.8 1−0.04
−0.02
0
0.02
0.04
0.06
0 0.2 0.4 0.6 0.8 1−0.06
−0.04
−0.02
0
0.02
0.04
0 0.2 0.4 0.6 0.8 1−0.04
−0.02
0
0.02
0.04
0.06
0 0.2 0.4 0.6 0.8 1−0.04
−0.02
0
0.02
0.04
0 0.2 0.4 0.6 0.8 1−0.4
−0.2
0
0.2
0.4
0 0.2 0.4 0.6 0.8 1−0.2
−0.1
0
0.1
0.2
0 0.2 0.4 0.6 0.8 1
−0.05
0
0.05
0 0.2 0.4 0.6 0.8
−0.04
−0.02
0
0.02
0.04
0.06
0.08
0 0.2 0.4 0.6 0.8 1−0.05
0
0.05
0 0.2 0.4 0.6 0.8 1−0.04
−0.02
0
0.02
0.04
0.06
0 0.2 0.4 0.6 0.8 1−0.06
−0.04
−0.02
0
0.02
0.04
0 0.2 0.4 0.6 0.8 1−0.04
−0.02
0
0.02
0.04
0.06
0 0.2 0.4 0.6 0.8 1−0.04
−0.02
0
0.02
0.04
0 0.2 0.4 0.6 0.8 1−0.4
−0.2
0
0.2
0.4
0 0.2 0.4 0.6 0.8 1−0.2
−0.1
0
0.1
0.2
0 0.2 0.4 0.6 0.8 1
−0.05
0
0.05
0 0.2 0.4 0.6 0.8
−0.04
−0.02
0
0.02
0.04
0.06
0.08
0 0.2 0.4 0.6 0.8 1−0.05
0
0.05
0 0.2 0.4 0.6 0.8 1−0.04
−0.02
0
0.02
0.04
0.06
0 0.2 0.4 0.6 0.8 1−0.06
−0.04
−0.02
0
0.02
0.04
0 0.2 0.4 0.6 0.8 1−0.04
−0.02
0
0.02
0.04
0.06
0 0.2 0.4 0.6 0.8 1−0.04
−0.02
0
0.02
0.04
Figure 10: The first nine spatial basis (u1(x, t), u2(x, t), · · · , u9(x, t) from the left to the right and from thetop to the bottom) computed by DyBO (blue curves) at time t = 1.0. Red curves are given by the gPCmethod.
Yj = Haj
i = 1, 2, · · · ,mj
β
E [YHβ]
α∈J
(a)
2 4 6 8 10 12
10
20
30
40
50
60
70
80
2 4 6 8 10 12
10
20
30
40
50
60
70
80
2 4 6 8 10 12
10
20
30
40
50
60
70
80−0.6
−0.4
−0.2
0
0.2
0.4
0.6
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
−0.2
−0.1
0
0.1
0.2
0.3
(b) DyBO
2 4 6 8 10 12
10
20
30
40
50
60
70
80
2 4 6 8 10 12
10
20
30
40
50
60
70
80
2 4 6 8 10 12
10
20
30
40
50
60
70
80−0.6
−0.4
−0.2
0
0.2
0.4
0.6
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
−0.2
−0.1
0
0.1
0.2
0.3
(c) gPC
Figure 11: Stochastic basis HA computed by DyBO at time t = 1.0.
25
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−0.4
−0.3
−0.2
−0.1
0
0.1
0.2
0.3
0.4
x
Mea
n
Initial mean
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
x
ST
D
Initial STD
Figure 12: Time history of the mean (left) and STD (right) of the stochastic Burgers equation for t ≤ 6.0.
6. Summary
In this paper, we have proposed and developed the DyBO method for a class of time-dependent SPDEs,whose solutions enjoy a low-dimensional structure in the sense of KL expansions. Unlike other traditionalmethods, such as MC, qMC, gPC, gSC, our DyBO method explores the inherent low-dimensional structureof the stochastic solution and essentially tracks the KL expansion dynamically. Thus, without the need ofperforming additional post-processing, the DyBO methods reveal directly the intrinsic low-dimensional struc-tures/dynamics of related physical processes. We have proved rigorously the preservation of bi-orthogonalityin DyBO methods and verified it numerically in several examples. Depending on the numerical represen-tations of the stochastic basis Y, three versions of DyBO methods have been proposed, i.e., DyBO-MC,DyBO-gSC, DyBO-gPC. We have primarily focused on DyBO-gPC in this paper.
By exploring the low-dimensional structure of the solution, our DyBO methods offer considerable com-putational savings over other traditional methods. An important advantage of DyBO methods over otherreduced basis methods is that we construct the sparsest set of spatial basis on the fly without invoking anyexpensive offline computations. From the perspective of gPC/gSC methods, our DyBO methods automat-ically use the linear combinations of polynomial chaos basis as the best set of stochastic basis on the flywithout introducing any heuristics to select multi-indices.
To the best of our knowledge, the DyBO method presented in this paper is the first systematic attempt todirectly target KL expansions and fully explore the bi-orthogonality. To make DyBO a feasible computationalmethod, we have overcome several challenges in both theory and numerical implementations. From thetheoretical consideration, one of the main difficulties is how to eliminate the extra degrees of freedom inducedby allowing both the spatial and the stochastic basis to change in time. From numerical implementationpoint of view, the adoptation of tensor/matrix notations enables us to present the DyBO formations in avery concise form. The vector/matrix notations also greatly simplify the codings and allow us to proceed inan intuitive way, especially for object-oriented programming languages. Temporarily freezing spatial basisU or stochastic basis Y was proposed to deal with the special moments when eigenvalues cross each other.
There are still some limitations of the DyBO method in its present form. The most important issue ishow to deal with multiscale stochastic PDEs. When small scale features are important in our problem, thenumber of effective modes required to resolve the stochastic solution increases inversely proportional to thesmallest correlation length. In this case, the effective dimension of the stochastic problem is large even if weuse the KL expansion. We are currently investigating a multiscale version of the DyBO method. The mainidea is to combine multiscale techniques [2, 20, 27] in solving deterministic PDE with the DyBO methodto further reduce the computational complexity for solving multiscale SPDEs. The result will be presentedin a subsequent paper. Another issue is to explore the freedom in choosing the dynamic basis to avoid thenumerical difficulty associated with eigenvalue crossing. By working on an equivalent basis, we could workon a transformed space in which we can derive an evolution system without suffering from the degeneracyimposed by bi-orthogonality. By tracking this mapping dynamically, we could restore the bi-orthogonal
26
basis on the fly. Finally, the success of our method depends critically on our ability to adaptively add orremove modes dynamically. This is one of the main topics in the second part of our paper.
Acknowledgement
This work was supported by AFOSR MURI Grant FA9550-09-1-0613, a DOE grant DE-FG02-06ER25727,and a NSF grant DMS-0908546.
Appendix A. Derivation of the DyBO Formulation
In this appendix, we will complete the derivation of the DyBO formulation which we began in Section2. We continue our derivation starting from Eq. (11). Note that Eq. (11) gives an implicit evolution systemfor the mean solution u, the spatial basis U and the stochastic basis Y of the KL expansion. On the otherhand, since ∂U
∂t and dYdt appear on both sides of Eq. (11), we cannot use it to update the spatial basis U
and the stochastic basis Y. In this subsection, we demonstrate that the partial anti-symmetrization Q of⟨UT , ∂U∂t
⟩in Eq. (11b) and the anti-symmetrization Q of E
[YT dY
dt
]in Eq. (11c) do produce an equivalent
system to the system (11) as long as U and Y are bi-orthogonal. This can be seen by taking the temporalderivative of orthogonality conditions (5a) and (5b),
d
dt
⟨UT , U
⟩=
⟨dUT
dt, U
⟩+
⟨UT ,
dU
dt
⟩,
d
dtE[YTY
]= E
[dYT
dtY
]+ E
[YT dY
dt
].
Obviously, the off-diagonal elements of both matrices ddt
⟨UT , U
⟩and d
dtE[YTY
]are zeros due to the
bi-orthogonality condition (5), which in turn implies both matrices⟨
dUT
dt , U⟩
and E[
dYT
dt Y]
are invari-
ant under partial anti-symmetrization Q. Furthermore, Y being orthonormal implies E[
dYT
dt Y]
is anti-
symmetric. On the other hand, bi-orthogonality (5) is preserved if it is satisfied initially at t = 0 and suchinvariance is satisfied at any later time t > 0. Thus, we conclude that the bi-orthogonality condition (5) ispreserved for all time if and only it is true initially and the following conditions hold for any t > 0:
Q(⟨
UT ,∂U
∂t
⟩)=
⟨UT ,
∂U
∂t
⟩, (A.2a)
Q(E[YT dY
dt
])= E
[YT dY
dt
]. (A.2b)
In other words, anti-symmetrization enforces the bi-orthogonality condition, which essentially characterizesKL expansions. After applying this result to the system (11), we arrive at
∂u
∂t= E [Lu] , (A.3a)
∂U
∂t= UΛ−1
U Q(⟨
UT ,∂U
∂t
⟩)+GU(u,U,Y), (A.3b)
dY
dt= YQ
(E[YT dY
dt
])+GY(u,U,Y). (A.3c)
Next, we will eliminate the time derivatives from the right hand sides in Eq. (A.3b) and Eq. (A.3c). Wefirst observe that Eq. (A.3b) does not determine ∂U
∂t uniquely. In fact, ∂U∂t −UΛ−1
U Q(⟨
UT , ∂U∂t⟩)
is the
projection of ∂U∂t into the orthogonal complementary set of U. Thus, we cannot determine the projection of
∂U∂t into the space spanned by U from Eq. (A.3b). Notice that (U, U) forms a complete orthogonal basis
27
in the physical space for any given time. To determine the projection of ∂U∂t in the space spanned by U, we
express ∂U∂t in terms of the complete orthogonal basis given by (U, U):
∂U
∂t= UC + UC, (A.4)
where C(t) ∈ Rm×m and C(t) ∈ R∞×m . We will show that UC = GU(u,U,Y). We note that UC isthe projection of ∂U
∂t into U. The matrix C represents the extra degrees of freedom in evolving ∂U∂t . We
will determine these extra degrees of freedom by enforcing the bi-orthogonal condition and a compatibilitycondition. Note that⟨
UT ,∂U
∂t
⟩=⟨UT , UC + UC
⟩=⟨UT , U
⟩C +
⟨UT , U
⟩C =
⟨UT , U
⟩C. (A.5)
Substituting Eq. (A.5) and Eq. (A.4) into Eq.(A.3b) gives
U(C−Λ−1
U Q (ΛUC))
= GU(u,U,Y)− UC. (A.6)
Recall that the left side of Eq. (A.6) is in span(U) ⊂ L2 (D) while the right side is in its orthogonalcomplement. We obtain
C−Λ−1U Q (ΛUC) = 0, (A.7a)
UC = GU(u,U,Y). (A.7b)
Similarly, the change of the stochastic basis, dYdt , can be written in the form of
dY
dt= YD + YD, (A.8)
where D(t) ∈ Rm×m and D(t) ∈ R∞×m. We will show that YD = GY(u,U,Y). We note that YD is theprojection of dY
dt into the space spanned by Y. The matrix D represents the extra degrees of freedom in
evolving dYdt . First, we observe that
E[YT dY
dt
]= E
[YTY
]D + E
[YT Y
]D = D. (A.9)
Substituting Eq. (A.9) and Eq. (A.8) into Eq.(A.3c), we get
Y (D−Q (D)) = GY(u,U,Y)− YD. (A.10)
Note that the left side of Eq. (A.10) is in span(Y) ⊂ L2 (Ω) and the right side is in its orthogonal complement.This gives rise to the following equations:
D−Q (D) = 0, (A.11a)
YD = GY(u,U,Y). (A.11b)
However, Eq. (A.7a) and Eq. (A.11a) are not sufficient to determine matrices C and D. To find additionalequations for C and D, we substitute Eq. (A.4) and Eq. (A.8) back into the original stochastic partialdifferential equation (1a) and get
U(DT + C
)YT + UDT YT + UCYT = Lu. (A.12)
Multiplying UT from the left and Y from the right on both sides of Eq. (A.12) and taking inner products〈·, ·〉 and expectations E [·], we obtain the following compatibility condition for C and D:
DT + C = G∗(u,U,Y), (A.13)
where we have used (5a), (5b) and G∗(u,U,Y) = Λ−1U
⟨UT , E
[LuY
]⟩∈ Rm×m. Using Eq. (A.7a),
Eq. (A.11a), and the compatibility condition (A.13), we can determine C and D uniquely provided that〈ui, ui〉 6= 〈uj , uj〉 for i 6= j. This completes our derivation of the DyBO formulation.
28
Appendix B. Derivation of the DyBO-gPC Formulation of Stochastic Burgers Equation
In this appendix, we provide a detailed derivation of Eq. (53). Consider the m-term truncated KLexpansion of the stochastic Burgers equation solution u = u + UYT = u + UATHT . Simple calculationsgive
Lu = −(u+ UYT
)(∂u∂x
+∂U
∂xYT
)+ ν
(∂2u
∂x2+∂2U
∂x2YT
)+ f
= Lu+
(ν∂2U
∂x2− ∂ (uU)
∂x
)YT −UYTY
∂UT
∂x+ f.
Thus, we have
E [Lu] = Lu−U∂UT
∂x,
where we have used E[YTY
]= I, E [Y] = 0, and E [f ] = 0. Direct calculations give
Lu =
(ν∂2U
∂x2− ∂ (uU)
∂x
)YT −UYTY
∂UT
∂x+ U
∂UT
∂x+ f,
E[LuY
]=
(ν∂2U
∂x2− ∂ (uU)
∂x
)+ E [fY]− E
[UYTY
∂UT
∂xY
],
⟨Lu, U
⟩= Y
⟨ν∂2UT
∂x2−∂(uUT
)∂x
, U
⟩+ 〈f, U〉+
⟨U∂UT
∂x, U
⟩−⟨
UYTY∂UT
∂x, U
⟩,
ΛUG∗(u,U,Y) =
⟨UT , ν
∂2U
∂x2− ∂ (uU)
∂x
⟩+⟨UT , E [fY]
⟩−⟨
UT , E[UYTY
∂UT
∂xY
]⟩,
where the last terms on the right hand sides of the last three equations can be written in component forms,for i, j, k = 1, 2, · · · ,m,
E[UYTY
∂UT
∂xY
]=
(E[uiYiYj
∂uj∂x
Yk
])1k
=
(ui∂uj∂x
E [YiYjYk]
)1k
,⟨UYTY
∂UT
∂x, U
⟩=
(⟨uiYiYj
∂uj∂x
, uk
⟩)1k
=
(YiYj
⟨ui∂uj∂x
, uk
⟩)1k
,⟨UT , E
[UYTY
∂UT
∂xY
]⟩=
(⟨ui∂uj∂x
, ul
⟩E [YiYjYk]
)lk
,
where (·)1k and (·)lk emphasize row vector and m-by-m matrix, respectively. We write third-order m-by-m-by-m tensors
T(Y)ijk = E [YiYjYk] and T
(U)ijl =
⟨ui∂uj∂x
, ul
⟩i, j, k, l = 1, 2, · · · ,m.
Clearly, T(Y) is symmetric with respect to all three indices, i.e., T(Y)ijk = T
(Y)perm(ijk) for ∀i, j, k ∈ 1, 2, · · · ,m,
where perm (ijk) is any permutation of indices i, j, k, and T(U) is symmetric with respect to the first
and the third indices, i.e., T(U)ijk = T
(U)kji for ∀i, k ∈ 1, 2, · · · ,m. Such symmetries can be explored in
numerical implementations to achieve further computational reductions. Using Eq. (16), we obtain the
29
DyBO formulation,
∂u
∂t= Lu−U
∂UT
∂x, (B.1a)
∂U
∂t= −UDT +
(ν∂2U
∂x2− ∂ (uU)
∂x
)−(ui∂uj∂x
T(Y)ijk
)1k
+ E [fY] , (B.1b)
dY
dt= Y
(−CT +
⟨ν∂2UT
∂x2−∂(uUT
)∂x
, U
⟩Λ−1
U
)+(T
(U)iik
)1k
Λ−1U
−(YiYjT
(U)ijk
)1k
Λ−1U + 〈f, U〉Λ−1
U , (B.1c)
where
ΛUG∗(u,U,Y) =
⟨UT , ν
∂2U
∂x2− ∂ (uU)
∂x
⟩−(T
(U)ijl T
(Y)ijk
)lk
+⟨UT , E [fY]
⟩. (B.2)
Similarly, the DyBO-gPC formulation can be obtained by substituting Y = HA into (B.1)(B.2) andusing the face E
[YTY
]= ATA = I.
References
[1] I. Babuska, F. Nobile, and R. Tempone. A stochastic collocation method for elliptic partial differential equations withrandom input data. SIAM J. Numer. Anal., 45(3):1005, 2007.
[2] A. Bensoussan, J.-L. Lions and G. Papanicolaou, Asymptotic Analysis for Periodic Structure, North-Holland, Amsterdam,1978.
[3] G. Berkooz, P. Holmes, and J.L. Lumley. The proper orthogonal decomposition in the analysis of turbulent flows. Ann.Rev. Fluid Mech., 25:539-575, 1993.
[4] L. Borcea, G. Papanicolaou, C. Tsogka and J. Berryman. Imaging and time reversal in random media. Inverse Problems,18:1247-1279, 2002.
[5] L. Borcea, G. Papanicolaou, and C. Tsogka. Theory and applications of time reversal and interferometric imaging. InverseProblems, 19:139-164, 2003.
[6] J.P. Bouchaud, M. Mezard, and G. Parisi. Scaling and intermittency in Burgers turbulence. Physical Review E1,52(4):3656-3674, 1995.
[7] S. Boyaval, C. LeBris, T. Lelievre, Y. Maday, N. Nguyen, and A. Patera. Reduced basis techniques for stochastic problems,Arch. Comput. Meth. Eng., 17 (2012), 435-454.
[8] H.J. Bungartz and M. Griebel. Sparse grids. Acta Numerica, 13:147-269, 2004.[9] R.H. Cameron and W.T. Martin. The orthogonal developement of non-linear functionals in series of Fourier-Hermite
Functionals. Ann. Math., 48(2):385-392, 1947.[10] M.L. Cheng. Adaptive methods exploring intrinsic sparse structures of stochastic partial differential equations. Doctroal
thesis, California Institute of Technology, 2012.[11] M.L. Cheng, T.Y. Hou, Z.W. Zhang. A dynamically bi-orthogonal method for time-dependent stochastic partial differential
equations II: adaptivity and generalizations. Submitted.[12] A. J. Chorin. Hermite expansion in Monte-Carlo simulations. J. Comput. Phys., 8:472-482, 1971.[13] A. J. Chorin. Gaussian fields and random flow. J. Fluid Mech., 63:21-32, 1974.[14] A. J. Chorin and X.M. Tu. Implicit sampling for particle filters. Proc. Nat. Acad. Sc., 106:17249-17254, 2009.[15] M. Dashti and A.M. Stuart, Uncertainty quantification and weak approximation of an elliptic inverse problem. SIAM J.
Numer. Anal., 49:2524-2542, 2011.[16] A. Doostan and H. Owhadi. A non-adapted sparse approximation of PDEs with stochastic inputs. J. Comput. Phys.,
230(8):3015-3034, 2011.[17] P. Dostert, Y. Efendiev, T. Y. Hou, and W. Luo, Coarse gradient Langevin algorithms for dynamic data integration and
uncertainty quantification. J. Comput. Phys., 217:123–142, 2006.[18] W.E, K. Khanin, A. Mazel, and Y. Sinai. Invariant measures for Burgers equation with stochastic forcing. Ann. Math.,
151(3):877-960, 2000.[19] Y. Efendiev, T. Y. Hou, and W. Luo, Preconditioning of Markov Chain Monte Carlo simulations using coarse-scale
models. SIAM J. Sci. Comput., 28(2):776-803, 2006.[20] Y. Efendiev and T. Y. Hou, Multiscale Finite Element Methods: Theory and Applications. Springer, New York, 2009.[21] Y. Efendiev, J. Galvis, and T. Y. Hou, Generalized Multiscale Finite Element Methods (GMsFEM), submitted to J.
Comput. Phys., 2012.[22] J. N. Franklin. Matrix Theory. Dover Books on Mathematics. Dover Publications, First edition, 2000.[23] R.G. Ghanem and P.D. Spanos. Stochastic Finite Elements: A Spectral Approach. New York : Springer-Verlag, 1991.[24] M.B. Giles. Multilevel Monte Carlo path simulation. Operations Research, 56(3):607-617, 2008.
30
[25] W. Hackbusch. A Sparse Matrix Arithmetic Based on H-Matrices. Part I: Introduction to H-Matrices. Computing,62(2):89-108, 1999.
[26] I. Hosokawa and K. Yamamoto. Turbulence in randomly force, one-dimensional Burgers flow. J. Stat. Phys., 13(3):245-272,1975.
[27] T.Y. Hou and X.H. Wu, A multiscale finite element method for elliptic problems in composite materials and porous media.J. Comput. Phys., 134:169-189, 1997.
[28] T. Y. Hou, W. Luo, B. Rozovskii, and H. Zhou. Wiener Chaos expansions and numerical solutions of randomly forcedequations of fluid mechanics. J. Comput. Phys., 216(2):687-706, 2006.
[29] K. Karhunen. Uber lineare Methoden in der Wahrscheinlichkeitsrechnung. Ann. Acad. Sci. Fennicae. Ser. A. I. Math.-Phys.37:1-79, 1947.
[30] P. E. Kloeden and E. Platen. Numerical Solution of Stochastic Differential Equations, Springer-Verlag Berlin, Heidelberg,1992.
[31] L.Y. Li, H.A. Tchelepi, and D.X. Zhang. Perturbation-based moment equation approach for flow in heterogeneous porousmedia: applicability range and analysis of high-order terms. J Compu. Phys., 188(1):296-317, 2003.
[32] M. Loeve, Probability Theory. Vol. II, 4th ed. GTM. 46. Springer-Verlag. ISBN 0-387-90262-7.[33] W. Luo. Wiener chaos expansion and numerical solutions of stochastic partial differential equations. Ph.D. thesis, Cali-
fornia Institute of Technology, 2006.[34] X. Ma and N. Zabaras. An adaptive hierarchical sparse grid collocation algorithm for the solution of stochastic differential
equations. J.Comput. Phys., 228(8):3084-3113, 2009.[35] Y. Maday, Reduced-basis method for the rapid and reliable solution of partial differential equations. In: Proceedings of
international conference of mathematicians, Madrid. European Mathematical Society, Zurich, 2006.[36] O. Le Maitre. Uncertainty propagation using Wiener-Haar expansions. J. Comput. Phys., 197(1):28-57, 2004.[37] O. Le Maitre. A Newton method for the resolution of steady stochastic Navier-Stokes equations. Computers and Fluids,
38(8):1566-1579, 2009.[38] A. J. Majda. I. Timofeyev. and E. V. Eijnden. A mathematical framework for stochastic climate models. CPAM, 54:891-
974, 2001.[39] A. J. Majda, C. Franzke and B. Khouider. An applied mathematics perspective on stochastic modelling for climate. Phil.
Trans. Roy. Soc., 366:2427-2453, 2008.[40] A. J. Majda and M. Branicki. Lessons in Uncertainty Quantification for Turbulent Dynamical System. DCDS-A, 32:3133-
3221, 2012.[41] W.J. Morokoff and R.E. Caflisch. Quasi-sandom sequences and their discrepancies. SIAM J Sci. Comput., 15(6):1251-1279,
1994.[42] H.N. Najm. Uncertainty quantification and polynomial chaos techniques in computational fluid dynamics. Ann. Rev. Fluid
Mech,, 41(1):35-52, 2009.[43] H. Niederreiter Random number generation and quasi-Monte Carlo methods. SIAM Publ., 1992.[44] H. Owhadi, C. Scovel, T. J. Sullivan, M. McKerns, and M. Ortiz. Optimal uncertainty quantification, SIAM Review, to
appear, 2012, arXiv:1009.0679.[45] G. Rozza, D. B. P Huynh, and A. T. Patera, Reduced basis approximation and a posteriori error estimation for affinely
parametrized elliptic coercive partial differential equations. Application to transport and continuum mechanics. ArchComput Methods Eng 15(3):229-275, 2008.
[46] T.P. Sapsis and P.F.J. Lermusiaux. Dynamically orthogonal field equations for continuous stochastic dynamical systems.Physica D, 238:2347-2360, 2009.
[47] T.P. Sapsis and P.F.J. Lermusiaux. Dynamical criteria for the evolution of the stochastic dimensionality in flows withuncertainty. Physica D, 241(1):60-76, 2012.
[48] C. Schwab, and R.A. Todor,. Karhunen-Loeve approximation of random fields by generalized fast multipole methods. J.Comput. Phys., 217:100-122, 2006.
[49] D. Venturi. On proper orthogonal decomposition of randomly perturbed fields with applications to flow past a cylinder andnatural convection over a horizontal plate. J. Fluid Mech., 559:215-254, 2006.
[50] D. Venturi, X. Wan, and G.E. Karniadakis. Stochastic low-dimensional modelling of a random laminar wake past acircular cylinder. J. Fluid Mech., 606:339-367, 2008.
[51] X. Wan and G.E. Karniadakis. An adaptive multi-element generalized polynomial chaos method for stochastic differentialequations. J.Comput. Phys., 209(2):617-642, 2005.
[52] N. Wierner. The homogeneous chaos. Amer. J. Math., 60:897-936, 1938.[53] X.-H. Wu, L. Bi, and S. Kalla, Effective parametrization for reliable reservoir performance predictions. Internat. J. for
Uncertainty Quantification, 2(3):259-278, 2012.[54] D. Xiu and G.E. Karniadakis. The Wiener-Askey polynomial chaos for stochastic differential equations. SIAM J. Sci.
Comput., 24(2):614-644, 2002.[55] D. Xiu and G.E. Karniadakis. Modeling uncertainty in flow simulations via generalized polynomial chaos. J. Comput.
Phys., 187(1):137-167, 2003.
31