new numerical approaches for discrete fracture network...
TRANSCRIPT
Intro Model Discretization Numerical Results Large scale DFNs
New numerical approaches for Discrete Fracture Network flowsimulations
Stefano Berrone
Dipartimento di Scienze Matematiche“ Giuseppe Luigi Lagrange ”Politecnico di [email protected]
joint work withMatias Benedetto, Andrea Borio, Sandra Pieraccini, Stefano Scialo,
Fabio Vicini
February 24, 2015Politecnico di Milano
Stefano Berrone Simulations of fluid flows in DFNs with non-conforming grids 1
Intro Model Discretization Numerical Results Large scale DFNs
DFN
Figure : Example of DFN
Discrete fracture network models forflows simulations:
3D network of intersectingfractures
Fractures are represented asplanar polygons
Rock matrix is consideredimpervious - flow confined in theDFN
Flow driven by hydraulic headgradientss modeled by Darcylaw in the fractures
Flux balance and hydraulic headcontinuity imposed across traceintersections
Challenges:
Complex domain: difficulties in good quality mesh generation
Stefano Berrone Simulations of fluid flows in DFNs with non-conforming grids 2
Intro Model Discretization Numerical Results Large scale DFNs
Notation
Fi ⊂ R3, i ∈ I is a generic fracture of the system.
The computational domain is denoted Ω =⋃
i∈I Fi, ∂Ω =⋃
i∈I ∂Fi.
Fracture intersections are called traces and denoted by S, S ∈ S.
Each trace is shared by exactly two fractures: S = Fi ∩ Fj, andIS = i, j.Si is the set of traces on fracture Fi while S denotes the set of all thetraces.
Stefano Berrone Simulations of fluid flows in DFNs with non-conforming grids 3
Intro Model Discretization Numerical Results Large scale DFNs Fracture model PDE Optimization
Single fracture model
Let ∂Fi = ΓiN ∪ ΓiD with ΓiN ∩ ΓiD = ∅ and ΓiD 6= ∅.Find Hi ∈ H1
D(Fi) such that:
(Ki∇Hi,∇v) = (qi, v) + 〈GiN, v|ΓiN〉
H−12 (ΓiN),H
12 (ΓiN)
, ∀v ∈ H10,D(Fi)
provides the hydraulic head Hi ∈ H1D(Fi).
Hi is the hydraulic head on the fracture Fi, H is the hydraulic head on Ω;Ki is the fracture transmissivity tensor: a symmetric and uniformlypositive definite tensor;∂Hi∂νi
= nti Ki∇Hi = GiN is the outward co-normal derivative of the
hydraulic head and ni the unit outward vector normal to the boundaryΓiN.
In the following[[∂Hi∂ν i
S
]]S
denotes the jump of the co-normal derivative along
the unique normal niS fixed for the trace S on the fracture Fi, this jump is
independent of the orientation of niS and is the net (incoming) flux provided by
the trace S to the fracture Fi.
Stefano Berrone Simulations of fluid flows in DFNs with non-conforming grids 4
Intro Model Discretization Numerical Results Large scale DFNs Fracture model PDE Optimization
Full fracture formulation
For each trace S ∈ S on the fracture Fi, let use denote by
USi :=
[[∂Hi
∂ν iS
]]S
USi ∈ US⊆H−
12 (S)
the flux entering in the fracture through the trace, and Ui ∈ USi the tuple offluxes US
i ∀S ∈ Si.Let us set ∂Fi = ΓiN ∪ ΓiD with ΓiN ∩ ΓiD = ∅ and ΓiD 6= ∅ (for ease ofexposition).Solving ∀i ∈ I the problem: find Hi ∈ H1
D(Fi) and Ui ∈ Si such that:
(Ki∇Hi,∇v) = (qi, v) + 〈Ui, v|Si〉USi ,USi ′
J(U)
+〈GiN, v|ΓiN〉
H−12 (ΓiN),H
12 (ΓiN)
, ∀v ∈ Vi = H10,D(Fi)
with additional conditions
Hi|S − Hj|S = 0, for i, j ∈ IS, ∀S ∈ S,
USi + US
j = 0, for i, j ∈ IS, ∀S ∈ S,
provides the hydraulic head H ∈ V = H1D(Ω).
Stefano Berrone Simulations of fluid flows in DFNs with non-conforming grids 5
Intro Model Discretization Numerical Results Large scale DFNs Fracture model PDE Optimization
Totally/Partially conforming meshes
Figure : Totally conforming mesh
Figure : Partially conforming mesh
Stefano Berrone Simulations of fluid flows in DFNs with non-conforming grids 6
Intro Model Discretization Numerical Results Large scale DFNs Fracture model PDE Optimization
non-conforming meshes: 36 fractures, 65 traces DFN
Figure : Non-conforming mesh on a 36 fracture DFN
Stefano Berrone Simulations of fluid flows in DFNs with non-conforming grids 7
Intro Model Discretization Numerical Results Large scale DFNs Fracture model PDE Optimization
PDE constrained optimization approach
Instead of solving the differential problems on the fractures coupled by thecorresponding matching conditions we look for the solution as the minimumof a PDE constrained quadratic optimal control problem, the variable Ubeing the control variable.Let us define the “observation” spaces
H12 (S) ⊆ HS(= US′), ∀S∈S, HSi =
∏S∈Si
HS, H =∏i∈I
HSi .
Let us define the differentiable functional J : U → R as
J(U) =∑S∈S
(||Hi(Ui)|S − Hj(Uj)|S ||
2HS + ||US
i + USj ||2US
)H(U)
We look for the control variable U providing the minimum of the functionalJ(U) constrained by the equation for Hi on each fracture.
Stefano Berrone Simulations of fluid flows in DFNs with non-conforming grids 8
Intro Model Discretization Numerical Results Large scale DFNs Fracture model PDE Optimization
Proposition
Let us define the spaces US andHS as
US = H−12 (S), HS = H
12 (S),
then the hydraulic head H ∈ H1D(Ω) giving the unique exact solution of DFN
problem satisfies the differential problems on Fi, ∀i ∈ I and corresponds toJ(U) = 0.
Stefano Berrone Simulations of fluid flows in DFNs with non-conforming grids 9
Intro Model Discretization Numerical Results Large scale DFNs Fracture model PDE Optimization
PDE constrained optimization approach
In order to remove the requirement of having a non-empty portion of Dirichletboundary on each fracture it is necessary to modify the definition of thecontrol variables on each trace as follows:
USi =
[[∂Hi
∂ν iS
]]S+αHi|S US
i ∈ US =H−12 (S), ∀S∈S
where α is a strictly positive scalar parameter. The definition of the functionalis modified accordingly as:
J(U) =∑S∈S
(||Hi(Ui)|S−Hj(Uj)|S ||
2HS
+F(Ki,Kj)||USi + US
j − α(Hi(Ui)|S − Hj(Uj)|S)||2US
)J(u)
The constraint equation on each fracture ∀i ∈ I becomes:find Hi ∈ H1
D(Fi) such that:
(Ki∇Hi,∇v) + α(
Hi|Si, v|Si
)Si
= (qi, v) + 〈Ui, v|Si〉USi ,USi ′
+〈GiN, v|ΓiN〉
H−12 (ΓiN),H
12 (ΓiN)
, ∀v ∈ Vi = H10,D(Fi) h(u)
Stefano Berrone Simulations of fluid flows in DFNs with non-conforming grids 10
Intro Model Discretization Numerical Results Large scale DFNs The discrete problem Numerical resolution
Definition of the discrete problem
Introduce a finite element triangulation on each fracture, completelyindependent of the triangulation on the intersecting fractures. Letus further define on this triangulation a finite element-like discretization hfor H.
Introduce also a discretization u for the control variable U, on the tracesof each fracture independently.
Let us choose US = HS = L2(S) for the discrete norms.
With arbitrary triangulations the minimum of the functional is not null.
Stefano Berrone Simulations of fluid flows in DFNs with non-conforming grids 11
Intro Model Discretization Numerical Results Large scale DFNs The discrete problem Numerical resolution
The problem under consideration is therefore the equality constrainedquadratic programming problem
minu,h
J(h, u) := 12 hTGhh− αhTBu + 1
2 uTGuu J(U)
s.t. Ah− B u = q H(U)
Gh ∈ RNF×NF, Gu ∈ RNT×NT
are symmetric positive semidefinite sparsematrices;
B ∈ RNF×NTis a sparse matrix;
vectors h ∈ RNFand u ∈ RNT
collect all DOFs for the hydraulic head onfractures and for the control variable on traces;
matrix A ∈ RNF×NFis the block diagonal stiffness matrix;
B ∈ RNF×NTis a “block diagonal” sparse matrix;
q ∈ RNFis a vector which accounts for possible source terms and
boundary conditions.
Stefano Berrone Simulations of fluid flows in DFNs with non-conforming grids 12
Intro Model Discretization Numerical Results Large scale DFNs The discrete problem Numerical resolution
The unconstrained problem
The first order optimality conditions for the constrained minimization problemare: Gh −αB AT
−αBT Gu −BT
A −B 0
hu−p
=
00q
.Use the linear constraint to remove h = A−1(B u + q) from J:
J(u) = J(u, h(u)) =12
uT(BT A−TGhA−1 B+Gu − αBT A−TB− αBTA−1 B)u
+qTA−T(GhA−1 B−αB)u =12
uTGu + uT q.
such that the minimum of the constrained problem is the same as the solutionto the unconstrained problem: min J(u)
This is equivalent to solve the global linear system
∇J(u) = Gu + q = 0.
Stefano Berrone Simulations of fluid flows in DFNs with non-conforming grids 13
Intro Model Discretization Numerical Results Large scale DFNs The discrete problem Numerical resolution
Back to local problems
Proposition
Matrix G = BT A−TGhA−1 B+Gu − αBT A−TB− αBTA−1 B is symmetricpositive definite.
Instead of solving the KKT system we solve the system Gu = −q with a PCGmethod performing the same operations.
Let us observe that, given a value to the control variables ui, ∀i ∈ I onlyLOCAL small independent s.p.d. linear systems on each fracture are solvedin order to evaluate the gradient.
Ah = B u + q,ATp = Ghh− αBu,
∇J(u) = BT p + Guu− αBTh.
Stefano Berrone Simulations of fluid flows in DFNs with non-conforming grids 14
Intro Model Discretization Numerical Results Large scale DFNs The discrete problem Numerical resolution
Resolution of the discrete problem
Algorithm
Preconditioned conjugate gradient method
Set k=0
Choose an initial guess uk
Compute hk, pk and gk = BT pk + Guuk − αBThk
Solve Pyk = gk
Set dk = −yk
While gk 6= 01 Compute λk with a line search along dk2 Compute uk+1 = uk + λkdk3 Update gk+1 = gk + λkGdk4 Solve Pyk+1 = gk+1
5 Compute βk+1 =gT
k+1yk+1
gTk yk
6 Update dk+1 = −yk+1 + βk+1dk7 k = k + 1
Stefano Berrone Simulations of fluid flows in DFNs with non-conforming grids 15
Intro Model Discretization Numerical Results Large scale DFNs The discrete problem Numerical resolution
We remark that the computation of gk+1 is clearly performed without formingmatrix G, but rather computing vector δgk
= Gdk through the following steps:1 Solve Aδh = B dk
2 Solve ATδp = Ghδh − αBdk
3 Compute δgk= BT δp + Gudk − αBTδh
Furthermore, since J is quadratic, the stepsize λk in Step 1 can be computedvia an exact line search:
λk =rT
k yk
dTk Gdk
.
The stepsize λk is therefore computed without much effort, as quantity Gdk isthe same needed in Step 3.
Stefano Berrone Simulations of fluid flows in DFNs with non-conforming grids 16
Intro Model Discretization Numerical Results Large scale DFNs The discrete problem Numerical resolution
Main advantages of the proposed method
the gradient method makes the optimization approach to DFNsimulations nearly inherently parallel ;
exchange of very small amount of data between processes;
each process only exchanges data with a limited number of other knownprocesses;
resolution of small linear systems independently performed;
the optimization approach is independent of the discretization approachused on the fractures.
Stefano Berrone Simulations of fluid flows in DFNs with non-conforming grids 17
Intro Model Discretization Numerical Results Large scale DFNs The discrete problem Numerical resolution
Circumventing mesh generation problems, different discretizations
The triangulation for the dis-crete solution is fully indepen-dent on each fracture and oneach trace.
Extended Finite Ele-ments: catch the ir-regular behaviour ofthe solution across thetraces improving accu-racy;
Figure : Enrichment functionaway from trace tip
−1
−0.5
0
0.5
1
−1−0.8−0.6−0.4−0.200.20.40.60.81
0
0.2
0.4
0.6
0.8
1
1.2
1.4
Figure : Tip enrichmentfunction
Stefano Berrone Simulations of fluid flows in DFNs with non-conforming grids 18
Intro Model Discretization Numerical Results Large scale DFNs The discrete problem Numerical resolution
Circumventing mesh generation problems
Figure : Example of non-conforming triangulation for the FEM-XFEM
Standard Finite Elements: less accurate around the traces, where thesolution has a sudden change of gradient. Nonetheless Standard FiniteElements can be used, maybe in conjunction with a mesh adaptivealgorithm, to refine the triangulation near the traces;eXtended Finite Elements: more accurate around the traces, where thesolution has a sudden change of gradient. A bit more complex;Virtual Element Method: starting from an arbitrary mesh requires a meshmodification. As accurate as XFEM.
Stefano Berrone Simulations of fluid flows in DFNs with non-conforming grids 19
Intro Model Discretization Numerical Results Large scale DFNs DFN Problems DFNs with Virtual Element Method
36 fractures, 65 traces DFN
Fractures size spans from 100m× 100m to 50m× 50m, different aspect ratios;
Figure : Solution with XFEM on a 36 fracture DFNStefano Berrone Simulations of fluid flows in DFNs with non-conforming grids 20
Intro Model Discretization Numerical Results Large scale DFNs DFN Problems DFNs with Virtual Element Method
36 fractures, 65 traces DFN
Figure : Solution with XFEM on a 36 fracture DFN
Stefano Berrone Simulations of fluid flows in DFNs with non-conforming grids 21
Intro Model Discretization Numerical Results Large scale DFNs DFN Problems DFNs with Virtual Element Method
120 fractures - 256 traces DFN
Size of fractures spans from 100m× 100m to 50m× 50m,different aspect ratios;
Stefano Berrone Simulations of fluid flows in DFNs with non-conforming grids 22
Intro Model Discretization Numerical Results Large scale DFNs DFN Problems DFNs with Virtual Element Method
Figure : 120F, Contour plot of the solution: detail
Stefano Berrone Simulations of fluid flows in DFNs with non-conforming grids 23
Intro Model Discretization Numerical Results Large scale DFNs DFN Problems DFNs with Virtual Element Method
Figure : 120F, Solution on the sourcefracture
Figure : 120F, Solution on the fracture 40
Stefano Berrone Simulations of fluid flows in DFNs with non-conforming grids 24
Intro Model Discretization Numerical Results Large scale DFNs DFN Problems DFNs with Virtual Element Method
Numerical results
LargeDFNs
Meshes of uSi and uS
j are given by the intersection of the independenttriangulations on the twin-fractures Fi and Fj with the trace S, respectively(different independent coarse meshes for uS
i and uSj ).
Φ =12
∑i∈Iin
∑Sm∈Si
∫Sm
(umi − αhi|Sm)−
∑i∈Iout
∑Sm∈Si
∫Sm
(umi − αhi|Sm)
,∆cons =
∣∣∣∑i∈Iin ∪Iout
∑Sm∈Si
∫Sm
(umi − αhi|Sm)
∣∣∣Φ
,
∆cont =
√∑Mm=1 ‖hi|Sm
− hj|Sm‖2
hmax∑M
m=1 |Sm|,
∆flux =
√∑Mm=1 ‖um
i + umj − α(hi|Sm
+ hj|Sm)‖2
Φ∑M
m=1 |Sm|,
Stefano Berrone Simulations of fluid flows in DFNs with non-conforming grids 25
Intro Model Discretization Numerical Results Large scale DFNs DFN Problems DFNs with Virtual Element Method
Results
Table : Results for DFNs 36F and 120F with Induced Node for u. XFEM and standardFEM compared.
XFEM FEMGrid u-dofs ∆cont ∆flux ∆cont ∆flux
36 fractures120 744 0.1363E-4 0.1596E-4 0.2536E-4 0.1740E-430 1292 0.1344E-4 0.1109E-4 0.1560E-4 0.1239E-47 2474 0.7623E-5 0.5183E-5 0.9475E-5 0.5347E-5
120 fractures120 2676 0.5610E-5 0.5570E-5 0.6866E-5 0.4544E-530 4616 0.3636E-5 0.2812E-5 0.4421E-5 0.2824E-57 8793 0.1875E-5 0.1517E-5 0.2496E-5 0.1703E-5
Stefano Berrone Simulations of fluid flows in DFNs with non-conforming grids 26
Intro Model Discretization Numerical Results Large scale DFNs DFN Problems DFNs with Virtual Element Method
DFNs with first order Virtual Element Method
Virtual Element Methods [1] can be easily ap-plied in conjuction with our optimization ap-proach to the DFNs models.Let Th be a decomposition of Fi in non overlap-ping polygons E with straight faces (differentnumber of edges are allowed). The discreteVirtual Finite Element space Vi,δ ⊂ H1
D(Fi) willbe defined element-wise, by introducing a localspace Vi,δ|E for each element E.
s ss
sss s
ppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppp
ppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppFigure : General P1 VEM elementE
For all E ∈ Th:
Vi,δ|E =
v ∈ H1(E) : ∆v = 0, v|e ∈ P1(e) ∀e ∈ ∂E.
the functions v ∈ Vi,δ|E are continuous (and known) on ∂E;
the functions v ∈ Vi,δ|E are unknown inside the element E;
P1(E) ⊆ Vi,δ|E.
[1] L. Beirao da Veiga, F. Brezzi, A. Cangiani, G. Manzini, L.D. Marini, A. Russo: Basic principles of
Virtual Element Methods, Math. Models Methods Appl. Sci. 23, (2013), 199-214.
Stefano Berrone Simulations of fluid flows in DFNs with non-conforming grids 27
Intro Model Discretization Numerical Results Large scale DFNs DFN Problems DFNs with Virtual Element Method
The bilinear form ah(·, ·)
For general continuous elements we set one degree of freedom for everyvertex of the mesh that represents the point-wise value of the solution atthe vertex.The global space Vi,δ is built assembling the local spaces Vi,δ|E as usual:
Vi,δ = v ∈ H10(Fi) : v|E ∈ Vi,δ|E ∀E ∈ Th
We have one dof per “non-Dirichlet” vertex.
A discrete bilinear form ai,δ(·, ·) is built element-wise
ai,δ(φ, ϕ) =∑
E∈Th
aEi,δ(φ, ϕ) ∀φ, ϕ ∈ Vi,δ,
whereaE
i,δ(·, ·) : Vi,δ|E × Vi,δ|E −→ R
are symmetric bilinear forms approximating aEi and satisfying a stability and
a consistency condition.
Stefano Berrone Simulations of fluid flows in DFNs with non-conforming grids 28
Intro Model Discretization Numerical Results Large scale DFNs DFN Problems DFNs with Virtual Element Method
The space discretization: VEM
Use flexibility of VEM in order to catch the behavior of the solution across thetraces, allowing for a partially conforming mesh, but still maintaining anindependent meshing process on each fracture.
1 Start from a given triangularmesh, built without taking intoaccount trace positions orconformity requirements.
2 Whenever a trace intersects oneelement edge, a new node iscreated. New nodes are alsocreated at trace tips. If the tracetip falls in the interior of anelement, the trace is prolongedup to the opposite mesh edge.
3 Intersected elements are thensplit into two new “sub-elements”,which become elements in theirown right. Convex polygons areobtained.
Stefano Berrone Simulations of fluid flows in DFNs with non-conforming grids 29
Intro Model Discretization Numerical Results Large scale DFNs DFN Problems DFNs with Virtual Element Method
The space discretization: VEM
Use flexibility of VEM in order to catch the behavior of the solution across thetraces, allowing for a partially conforming mesh, but still maintaining anindependent meshing process on each fracture.
1 Start from a given triangularmesh, built without taking intoaccount trace positions orconformity requirements.
2 Whenever a trace intersects oneelement edge, a new node iscreated. New nodes are alsocreated at trace tips. If the tracetip falls in the interior of anelement, the trace is prolongedup to the opposite mesh edge.
3 Intersected elements are thensplit into two new “sub-elements”,which become elements in theirown right. Convex polygons areobtained.
Stefano Berrone Simulations of fluid flows in DFNs with non-conforming grids 30
Intro Model Discretization Numerical Results Large scale DFNs DFN Problems DFNs with Virtual Element Method
The space discretization: VEM
Use flexibility of VEM in order to catch the behavior of the solution across thetraces, allowing for a partially conforming mesh, but still maintaining anindependent meshing process on each fracture.
1 Start from a given triangularmesh, built without taking intoaccount trace positions orconformity requirements.
2 Whenever a trace intersects oneelement edge, a new node iscreated. New nodes are alsocreated at trace tips. If the tracetip falls in the interior of anelement, the trace is prolongedup to the opposite mesh edge.
3 Intersected elements are thensplit into two new “sub-elements”,which become elements in theirown right. Convex polygons areobtained.
Stefano Berrone Simulations of fluid flows in DFNs with non-conforming grids 31
Intro Model Discretization Numerical Results Large scale DFNs DFN Problems DFNs with Virtual Element Method
Mesh smoothing
Figure : Left: VEM mesh without modification. Right: Same mesh after modifications.
Stefano Berrone Simulations of fluid flows in DFNs with non-conforming grids 32
Intro Model Discretization Numerical Results Large scale DFNs DFN Problems DFNs with Virtual Element Method
6 fracture DFN with the VEM
Fracture size from 7m× 7m to 4m× 4m, 6 traces
Figure : DFN: Solution and mesh
Stefano Berrone Simulations of fluid flows in DFNs with non-conforming grids 33
Intro Model Discretization Numerical Results Large scale DFNs DFN Problems DFNs with Virtual Element Method
6 fracture DFN with the VEM
Figure : VEM: VEM elements on fracture 2Figure : VEM: Solution on fracture 2
Stefano Berrone Simulations of fluid flows in DFNs with non-conforming grids 34
Intro Model Discretization Numerical Results Large scale DFNs DFN Problems DFNs with Virtual Element Method
6 fracture DFN with the VEM
Figure : VEM: VEM elements on fracture 6Figure : VEM: Solution on fracture 6
Stefano Berrone Simulations of fluid flows in DFNs with non-conforming grids 35
Intro Model Discretization Numerical Results Large scale DFNs
Figure : 3D view of DFN709 (left) and DFN1425 (right)
Stefano Berrone Simulations of fluid flows in DFNs with non-conforming grids 36
Intro Model Discretization Numerical Results Large scale DFNs
0 10 20 30 40 50 60 70 80 90
100
101
102
103
104
Angle[DEG]
# O
ccu
rren
ces
7091425
0 1 2 3 4 5
100
101
102
Angle[DEG]
# O
cc
urr
en
ce
s
7091425
Figure : DFN709 and DFN1425. Left: distribution of angles between pairs ofintersecting traces in the same fracture; right: zoom of lower values.
Stefano Berrone Simulations of fluid flows in DFNs with non-conforming grids 37
Intro Model Discretization Numerical Results Large scale DFNs
−2 −1 0 1 2 3
100
101
102
103
log10
(Length)
# O
cc
urr
en
ce
s
7091425
−4 −3 −2 −1 0 1 2
100
101
102
103
104
105
log10
(Length)
# O
ccu
rren
ces
7091425
Figure : DFN709 and DFN1425: distribution of trace lengths (left) and of distancesbetween couples of non-intersecting traces in the same fracture (right)
Stefano Berrone Simulations of fluid flows in DFNs with non-conforming grids 38
Intro Model Discretization Numerical Results Large scale DFNs
Table : Number of degrees of freedom for the hydraulic head h and the control variableu
NF NT
DFN δ = 0.5 δ = 2 δ = 7 δ = 0.5 δ = 2 δ = 7
709 1378796 356527 107467 363007 187645 105817878 1753469 452856 136549 329584 170733 96759915 1808368 467474 140799 351899 182739 103701
1425 2838981 733588 220980 1157310 598596 338489
Stefano Berrone Simulations of fluid flows in DFNs with non-conforming grids 39
Intro Model Discretization Numerical Results Large scale DFNs
Delta
Table : Relative residuals and flux conservation error ∆cons through iterations
relative residual ∆cons# iter δ = 0.5 δ = 2 δ = 7 δ = 0.5 δ = 2 δ = 7
DFN7091000 3.749e-4 4.826e-4 5.767e-4 5.864e-6 8.381e-4 1.056e-310000 1.014e-5 1.244e-5 1.845e-5 1.018e-5 1.031e-6 4.497e-520000 3.615e-6 4.715e-6 8.539e-6 4.814e-6 2.275e-6 2.177e-5
DFN8781000 3.716e-4 3.826e-4 3.944e-4 5.723e-5 1.925e-5 2.685e-510000 6.668e-6 9.271e-6 1.462e-5 2.083e-6 1.780e-6 5.027e-720000 4.821e-6 3.902e-6 4.705e-6 4.125e-6 2.091e-7 1.096e-6
DFN9151000 3.548e-4 3.840e-4 3.989e-4 4.932e-6 4.231e-5 1.962e-410000 8.486e-6 9.471e-6 1.636e-5 3.740e-6 8.943e-6 3.462e-620000 7.362e-6 5.896e-6 7.442e-6 2.729e-6 5.329e-6 5.204e-6
DFN14251000 4.022e-4 4.209e-4 4.424e-4 7.235e-4 7.678e-4 1.096e-310000 7.286e-6 8.687e-6 1.618e-5 6.258e-6 3.426e-5 5.754e-520000 2.527e-6 2.891e-6 6.787e-6 6.240e-6 3.683e-5 5.739e-5
Stefano Berrone Simulations of fluid flows in DFNs with non-conforming grids 40
Intro Model Discretization Numerical Results Large scale DFNs
Table : Hydraulic head mismatch ∆head and flux unbalance ∆flux through iterations
∆head ∆flux# iter δ = 0.5 δ = 2 δ = 7 δ = 0.5 δ = 2 δ = 7
DFN7091000 2.116e-6 2.356e-6 2.802e-6 5.683e-10 9.552e-10 1.002e-9
10000 5.138e-7 8.427e-7 1.092e-6 3.278e-10 3.461e-10 4.903e-1020000 5.002e-7 8.278e-7 1.062e-6 3.202e-10 3.409e-10 4.818e-10
DFN8781000 2.892e-6 3.322e-6 3.994e-6 4.863e-10 5.930e-10 7.395e-10
10000 7.318e-7 1.149e-6 1.668e-6 2.382e-10 3.299e-10 4.853e-1020000 7.030e-7 1.127e-6 1.641e-6 2.400e-10 3.319e-10 4.743e-10
DFN9151000 2.774e-6 3.187e-6 3.854e-6 4.397e-10 5.369e-10 6.780e-10
10000 7.595e-7 1.117e-6 1.683e-6 2.196e-10 3.045e-10 4.309e-1020000 7.109e-7 1.095e-6 1.641e-6 2.211e-10 3.004e-10 4.234e-10
DFN14251000 9.025e-7 1.009e-6 1.231e-6 1.311e-10 1.670e-10 2.102e-10
10000 2.168e-7 3.460e-7 5.397e-7 6.186e-11 9.671e-11 1.168e-1020000 2.121e-7 3.424e-7 5.284e-7 6.187e-11 9.553e-11 1.146e-10
Stefano Berrone Simulations of fluid flows in DFNs with non-conforming grids 41
Intro Model Discretization Numerical Results Large scale DFNs
Table : Number of communicated h-DOFs and u-DOFs for various numbers of parallelprocesses
DFN Slaves δ = 0.5 [%] δ = 2 [%] δ = 7 [%]
u DOFs comm
709
3 6978 [1.9] 3648 [1.0] 2119 [0.6]
1425
7 17755 [4.9] 9392 [2.6] 5451 [1.5]15 36994 [10.2] 19433 [5.4] 11253 [3.1]31 114061 [31.4] 59340 [16.3] 33779 [9.3]
3 55096 [4.8] 28694 [2.5] 16488 [1.4]7 108229 [9.4] 56773 [4.9] 32788 [2.8]
15 185821 [16.1] 97148 [8.4] 56057 [4.8]31 286723 [24.8] 149710 [12.9] 86405 [7.5]
h DOFs comm
709
3 7216 [0.5] 3885 [1.1] 2355 [2.2]
1425
7 18379 [1.3] 10016 [2.8] 6075 [5.7]15 38240 [2.8] 20682 [5.8] 12497 [11.6]31 117008 [8.5] 62290 [17.5] 36729 [34.2]
3 56671 [2.0] 30269 [4.1] 18066 [8.2]7 111724 [3.9] 60270 [8.2] 36284 [16.4]
15 191661 [6.8] 103000 [14.0] 61908 [28.0]31 295443 [10.4] 158446 [21.6] 95148 [43.1]
Stefano Berrone Simulations of fluid flows in DFNs with non-conforming grids 42
Intro Model Discretization Numerical Results Large scale DFNs
Table : Time per iteration of the parallel CG algorithm and efficiency for DFN709
Time [sec] Efficency %Version Slaves δ = 0.5 δ = 2 δ = 7 δ = 0.5 δ = 2 δ = 7
DFN709
Serial 1 0.83754 0.23432 0.08039
MPI
3 0.29631 0.08324 0.02909 94.22 93.83 92.11
MPI*
7 0.16898 0.04758 0.01651 70.81 70.35 69.5515 0.10830 0.03038 0.01057 51.56 51.41 50.6931 0.06239 0.01632 0.00568 43.31 46.33 45.68
15 0.11496 0.04281 0.01624 48.57 36.49 33.0031 0.07350 0.02281 0.00666 36.76 33.13 38.9663 0.03431 0.01194 0.00350 38.74 31.16 36.47
126 0.02243 0.00629 0.00233 29.64 29.56 27.35
Stefano Berrone Simulations of fluid flows in DFNs with non-conforming grids 43
Intro Model Discretization Numerical Results Large scale DFNs
Table : Time per iteration of the parallel CG algorithm and efficiency for DFN1425
Time [sec] Efficency %Version Slaves δ = 0.5 δ = 2 δ = 7 δ = 0.5 δ = 2 δ = 7
DFN1425
Serial 1 2.67725 0.70857 0.24254
MPI
3 0.91974 0.24591 0.08328 97.03 96.05 97.08
MPI*
7 0.48278 0.12251 0.04159 79.22 82.63 83.3015 0.28466 0.07176 0.02434 62.70 65.82 66.4431 0.14775 0.03888 0.02072 58.45 58.79 37.76
15 0.28194 0.07920 0.02947 63.31 59.64 54.8731 0.14638 0.04149 0.01434 59.00 55.09 54.5763 0.14838 0.04244 0.01540 28.64 26.50 25.00
127 0.06611 0.01881 0.00676 31.89 29.66 28.26
Stefano Berrone Simulations of fluid flows in DFNs with non-conforming grids 44
Intro Model Discretization Numerical Results Large scale DFNs
....................................................................................................................................................................................................................................................................
.....................................................................................................................................................................................................
...........................................................................................................................................................................................................
..................
..................
..................
..................
..................
..................
..................
..................
..................
..................
..................
..................
..................
...... ........................................................................................................................................................................................................................................................
....................
.................................................................................................................................................................................................
.....................................................................................................................................................................................................
........................................................................................................................................................................................................................................................
P1 P2 P3 P4
P0
Figure : Master-Slave communication scheme
Stefano Berrone Simulations of fluid flows in DFNs with non-conforming grids 45
Intro Model Discretization Numerical Results Large scale DFNs
pppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppp
pppppppppppppppppppp
pppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppp
pppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppp pppppppppppppppppppp
pppppppppppppppppppppppppppppppppppppppp
pppppppppppppppppppp pppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppp pppppppppppppppppppp pppppppppppppppppppp
pppppppppppppppppppp
pppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppp
pppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppp
P1
P3P2
P0
Figure : Point-to-Point communication scheme
Stefano Berrone Simulations of fluid flows in DFNs with non-conforming grids 46