parallel solution of the helmholtz equation with large wave numbers
DESCRIPTION
Parallel solution of the Helmholtz equation with large wave numbers. Dan Gordon Computer Science University of Haifa. Rachel Gordon Aerospace Eng. Technion. Outline of Talk. The Kaczmarz algorithm ( KACZ ) KACZ CARP ( C omponent- A veraged R ow P rojections) Applications of CARP - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Parallel solution of the Helmholtz equation with large wave numbers](https://reader035.vdocument.in/reader035/viewer/2022062315/568160df550346895dd00e8c/html5/thumbnails/1.jpg)
July 1, 2010 Parallel solution of the Helmholtz equation 1
Parallel solution of the Parallel solution of the Helmholtz equation with Helmholtz equation with
large wave numberslarge wave numbers
Dan GordonComputer ScienceUniversity of Haifa
Rachel GordonAerospace Eng.
Technion
![Page 2: Parallel solution of the Helmholtz equation with large wave numbers](https://reader035.vdocument.in/reader035/viewer/2022062315/568160df550346895dd00e8c/html5/thumbnails/2.jpg)
July 1, 2010 Parallel solution of the Helmholtz equation 2
Outline of TalkOutline of Talk The Kaczmarz algorithm (KACZ) KACZ CARP (Component-
Averaged Row Projections) Applications of CARPCARP-CG: CG acceleration of CARP Sample results with the Helmholtz
equation
![Page 3: Parallel solution of the Helmholtz equation with large wave numbers](https://reader035.vdocument.in/reader035/viewer/2022062315/568160df550346895dd00e8c/html5/thumbnails/3.jpg)
July 1, 2010 Parallel solution of the Helmholtz equation 3
KACZKACZ: The Kaczmarz algorithm: The Kaczmarz algorithm
Iterative method, due to Kaczmarz (1937). Rediscovered for CT as ART
Geometric algorithm: consider the hyperplane defined by each equation
Start from an arbitrary initial point Successively project current point
onto the next hyperplane, in cyclic order
![Page 4: Parallel solution of the Helmholtz equation with large wave numbers](https://reader035.vdocument.in/reader035/viewer/2022062315/568160df550346895dd00e8c/html5/thumbnails/4.jpg)
July 1, 2010 Parallel solution of the Helmholtz equation 4
KACZ: Geometric DescriptionKACZ: Geometric Description
eq. 1
eq. 2eq. 3
initialpoint
![Page 5: Parallel solution of the Helmholtz equation with large wave numbers](https://reader035.vdocument.in/reader035/viewer/2022062315/568160df550346895dd00e8c/html5/thumbnails/5.jpg)
July 1, 2010 Parallel solution of the Helmholtz equation 5
KACZKACZ with Relaxation Parameter with Relaxation ParameterKACZ can be used with a relaxation
parameter =1: project exactly on the
hyperplane<1: project in front of hyperplane>1: project beyond the hyperplane Cyclic relaxation: eq. i is assigned a
relaxation parameter i
![Page 6: Parallel solution of the Helmholtz equation with large wave numbers](https://reader035.vdocument.in/reader035/viewer/2022062315/568160df550346895dd00e8c/html5/thumbnails/6.jpg)
July 1, 2010 Parallel solution of the Helmholtz equation 6
Convergence Properties of Convergence Properties of KACZKACZ
KACZ with relaxation (0<<2) converges for consistent systems:– Herman, Lent & Lutz, 1978– Trummer, 1981
For inconsistent systems, KACZ converges cyclically:– Tanabe, 1971– Eggermont, Herman & Lent, 1981 (for
cyclic relaxation parameters).
![Page 7: Parallel solution of the Helmholtz equation with large wave numbers](https://reader035.vdocument.in/reader035/viewer/2022062315/568160df550346895dd00e8c/html5/thumbnails/7.jpg)
July 1, 2010 Parallel solution of the Helmholtz equation 7
Algebraic formulation of Algebraic formulation of KACZKACZ Given the system Ax = b Consider the "normal equations"
system AATy = b, x=ATy Well-known fact: KACZ is SOR
applied to the normal equations The relaxation parameter of KACZ is
the usual relax. par. of SOR
![Page 8: Parallel solution of the Helmholtz equation with large wave numbers](https://reader035.vdocument.in/reader035/viewer/2022062315/568160df550346895dd00e8c/html5/thumbnails/8.jpg)
July 1, 2010 Parallel solution of the Helmholtz equation 8
Block Mode & ParallelizationBlock Mode & Parallelization Block KACZ: projection onto affine
subspace defined by a block of eqns Block-sequential KACZ:
– partition eqns into blocks– each block consists of independent eqns– iterate over blocks– in each block, perform projections in
parallel
![Page 9: Parallel solution of the Helmholtz equation with large wave numbers](https://reader035.vdocument.in/reader035/viewer/2022062315/568160df550346895dd00e8c/html5/thumbnails/9.jpg)
July 1, 2010 Parallel solution of the Helmholtz equation 9
CARPCARP: Component-Averaged Row Projections: Component-Averaged Row Projections
A block-parallel version of KACZ The equations are divided into blocks
(not necessarily disjoint) Initial estimate: vector x=(x1,…,xn) Suppose x1 is a variable (component
of x) that appears in 3 blocks x1 is “cloned” as y1 , z1 , t1 in the
different blocks. Perform one (or more) KACZ
iteration(s) on each block (independently, in parallel)
![Page 10: Parallel solution of the Helmholtz equation with large wave numbers](https://reader035.vdocument.in/reader035/viewer/2022062315/568160df550346895dd00e8c/html5/thumbnails/10.jpg)
July 1, 2010 Parallel solution of the Helmholtz equation 10
CARPCARP – Explanation (cont) – Explanation (cont) The internal iterations in each block
produce 3 new values for the clones of x1 : y1’ , z1’ , t1’
The next iterative value of x1 is x1’ = (y1’ + z1’ + t1’)/3
The next iterate isx’ = (x1’ , ... , xn’)
Repeat iterations as needed for convergence
![Page 11: Parallel solution of the Helmholtz equation with large wave numbers](https://reader035.vdocument.in/reader035/viewer/2022062315/568160df550346895dd00e8c/html5/thumbnails/11.jpg)
July 1, 2010 Parallel solution of the Helmholtz equation 11
CARPCARP as Domain Decomposition as Domain Decomposition
x x
y0 1
1
domain Adomain A domain Bdomain B
external gridexternal gridpoint of Apoint of A
clone of clone of x1
Note: domains may overlap
![Page 12: Parallel solution of the Helmholtz equation with large wave numbers](https://reader035.vdocument.in/reader035/viewer/2022062315/568160df550346895dd00e8c/html5/thumbnails/12.jpg)
July 1, 2010 Parallel solution of the Helmholtz equation 12
Overview of Overview of CARPCARP
domain Adomain A domain Bdomain B
KACZKACZiterationsiterations
KACZKACZiterationsiterations
averagingaveraging
cloningcloning
KACZKACZ in superspace in superspace(with cyclic relaxation)(with cyclic relaxation)
![Page 13: Parallel solution of the Helmholtz equation with large wave numbers](https://reader035.vdocument.in/reader035/viewer/2022062315/568160df550346895dd00e8c/html5/thumbnails/13.jpg)
July 1, 2010 Parallel solution of the Helmholtz equation 13
Convergence of Convergence of CARPCARP Averaging Lemma: the component-
averaging and cloning operations of CARP are equivalent to KACZ row-projections in a certain superspace (with =1)
CARP is equivalent to KACZ in the superspace, with cyclic relaxation parameters – known to converge
![Page 14: Parallel solution of the Helmholtz equation with large wave numbers](https://reader035.vdocument.in/reader035/viewer/2022062315/568160df550346895dd00e8c/html5/thumbnails/14.jpg)
July 1, 2010 Parallel solution of the Helmholtz equation 14
CARPCARP Application: Solution of stiff Application: Solution of stiff linear systems from PDEslinear systems from PDEs
Elliptic PDEs w/large convection term result in stiff linear systems (large off-diagonal elements)
CARP is very robust on these systems, as compared to leading solver/preconditioner combinations
Downside: Not always efficient
![Page 15: Parallel solution of the Helmholtz equation with large wave numbers](https://reader035.vdocument.in/reader035/viewer/2022062315/568160df550346895dd00e8c/html5/thumbnails/15.jpg)
July 1, 2010 Parallel solution of the Helmholtz equation 15
CARPCARP Application: Electron Tomography Application: Electron Tomography(joint work with J.-J. Fernández)(joint work with J.-J. Fernández)
3D reconstructions: Each processor is assigned a block of consecutive slices. Data is in overlapping blobs.
The blocks are processed in parallel. The values of shared variables are
transmitted between the processors which share them, averaged, and redestributed.
![Page 16: Parallel solution of the Helmholtz equation with large wave numbers](https://reader035.vdocument.in/reader035/viewer/2022062315/568160df550346895dd00e8c/html5/thumbnails/16.jpg)
July 1, 2010 Parallel solution of the Helmholtz equation 16
CARP-CGCARP-CG: CG acceleration of : CG acceleration of CARPCARPCARP is KACZ in some superspace
(with cyclic relaxation parameters) Björck & Elfving (BIT 79): developed
CGMN, which is a (sequential) CG-acceleration of KACZ (double sweep, fixed relax. parameter)
We extended this result to allow cyclic relaxation parameters
Result: CARP-CG
![Page 17: Parallel solution of the Helmholtz equation with large wave numbers](https://reader035.vdocument.in/reader035/viewer/2022062315/568160df550346895dd00e8c/html5/thumbnails/17.jpg)
July 1, 2010 Parallel solution of the Helmholtz equation 17
CARP-CGCARP-CG: Properties: Properties Same robustness as CARP Very significant improvement in
performance on stiff linear systems derived from elliptic PDEs
Very competitive runtime compared to leading solver/preconditioner combinations on systems derived from convection-dominated PDEs
Improved performance in ET
![Page 18: Parallel solution of the Helmholtz equation with large wave numbers](https://reader035.vdocument.in/reader035/viewer/2022062315/568160df550346895dd00e8c/html5/thumbnails/18.jpg)
July 1, 2010 Parallel solution of the Helmholtz equation 18
CARP-CGCARP-CG: Properties: Properties On one processor, CARP-CG is
identical to CGMN Particularly useful on systems with
LARGE off-diagonal elements– example: convection-dominated PDEs
Discontinuous coefficients are handled without requiring domain decomposition (DD)
![Page 19: Parallel solution of the Helmholtz equation with large wave numbers](https://reader035.vdocument.in/reader035/viewer/2022062315/568160df550346895dd00e8c/html5/thumbnails/19.jpg)
July 1, 2010 Parallel solution of the Helmholtz equation 19
Robustness of CARP-CGKACZ inherently normalizes the eqns After normalization, the diagonal
elements of AAT are larger than the off-diagonal ones (in each row)
This is not diagonal dominance, but it makes the normal eqns manageable
Normalization was also found to be useful for discontinuous coefficients
![Page 20: Parallel solution of the Helmholtz equation with large wave numbers](https://reader035.vdocument.in/reader035/viewer/2022062315/568160df550346895dd00e8c/html5/thumbnails/20.jpg)
July 1, 2010 Parallel solution of the Helmholtz equation 20
The Helmholtz Equation Eqn: -Δu - k2u = f Wave length: = 2/k No. of grid pts per : Ng = 2/kh Shifted Laplacian approach:
– Bayliss, Goldstein & Turkel, 1983– Erlangga, Vuik & Oosterlee, 2004/06
-Δu – (ik2 u = fuses multigrid to solve the PC (PC = preconditioner)
![Page 21: Parallel solution of the Helmholtz equation with large wave numbers](https://reader035.vdocument.in/reader035/viewer/2022062315/568160df550346895dd00e8c/html5/thumbnails/21.jpg)
July 1, 2010 Parallel solution of the Helmholtz equation 21
The Helmholtz Equation Bollhöfer, Grote & Schenk, 2009:
introduced algebraic multilevel PC for the Helmholtz eqn in heterogeneous media. Uses symmetric max weight matchings and an inverse-based pivoting method.
Apologies to many other contributors to this problem!
![Page 22: Parallel solution of the Helmholtz equation with large wave numbers](https://reader035.vdocument.in/reader035/viewer/2022062315/568160df550346895dd00e8c/html5/thumbnails/22.jpg)
July 1, 2010 Parallel solution of the Helmholtz equation 22
ExperimentsCARP-CG was used with a fixed
relaxation parameter of 1.7 in all cases
Domain: unit square []
2nd order central difference scheme
![Page 23: Parallel solution of the Helmholtz equation with large wave numbers](https://reader035.vdocument.in/reader035/viewer/2022062315/568160df550346895dd00e8c/html5/thumbnails/23.jpg)
July 1, 2010 Parallel solution of the Helmholtz equation 23
Problem 1 (with analytic sol'n)
Based on Erlangga et al '04, §6.1 Eqn: (Δ+k2)u = (k2–52)sin(x)sin(2y) bndry condition: u=0 on all sides Analytic solution: u = sin(x)sin(2y) Grid points per : Ng = 6,8,10,12 No. of processors: 1 – 32 k = 100, 300
![Page 24: Parallel solution of the Helmholtz equation with large wave numbers](https://reader035.vdocument.in/reader035/viewer/2022062315/568160df550346895dd00e8c/html5/thumbnails/24.jpg)
![Page 25: Parallel solution of the Helmholtz equation with large wave numbers](https://reader035.vdocument.in/reader035/viewer/2022062315/568160df550346895dd00e8c/html5/thumbnails/25.jpg)
![Page 26: Parallel solution of the Helmholtz equation with large wave numbers](https://reader035.vdocument.in/reader035/viewer/2022062315/568160df550346895dd00e8c/html5/thumbnails/26.jpg)
![Page 27: Parallel solution of the Helmholtz equation with large wave numbers](https://reader035.vdocument.in/reader035/viewer/2022062315/568160df550346895dd00e8c/html5/thumbnails/27.jpg)
![Page 28: Parallel solution of the Helmholtz equation with large wave numbers](https://reader035.vdocument.in/reader035/viewer/2022062315/568160df550346895dd00e8c/html5/thumbnails/28.jpg)
July 1, 2010 Parallel solution of the Helmholtz equation 28
Problem 2 (homogeneous) Based on Erlangga et al '04, §6.2 Eqn: Δu + k2u = 0 Domain: unit square [0,1]x[0,1] Dirichlet bndry cond. on one side, with a
discontinuity at midpoint 1st-order absorbing bndry cond. on other
sides Grid points per : Ng = 6, 8, 10 No. of processors: 1 – 32 k = 75, 150, 300, 450, 600
![Page 29: Parallel solution of the Helmholtz equation with large wave numbers](https://reader035.vdocument.in/reader035/viewer/2022062315/568160df550346895dd00e8c/html5/thumbnails/29.jpg)
July 1, 2010 Parallel solution of the Helmholtz equation 29
![Page 30: Parallel solution of the Helmholtz equation with large wave numbers](https://reader035.vdocument.in/reader035/viewer/2022062315/568160df550346895dd00e8c/html5/thumbnails/30.jpg)
July 1, 2010 Parallel solution of the Helmholtz equation 30
![Page 31: Parallel solution of the Helmholtz equation with large wave numbers](https://reader035.vdocument.in/reader035/viewer/2022062315/568160df550346895dd00e8c/html5/thumbnails/31.jpg)
July 1, 2010 Parallel solution of the Helmholtz equation 31
![Page 32: Parallel solution of the Helmholtz equation with large wave numbers](https://reader035.vdocument.in/reader035/viewer/2022062315/568160df550346895dd00e8c/html5/thumbnails/32.jpg)
![Page 33: Parallel solution of the Helmholtz equation with large wave numbers](https://reader035.vdocument.in/reader035/viewer/2022062315/568160df550346895dd00e8c/html5/thumbnails/33.jpg)
Problem 3 (heterogeneous) 3-layer heterogeneous problem Based on Erlangga et al '04, §6.3 Everything is identical to Problem 2 EXCEPT:
k=600
k=450
k=300
![Page 34: Parallel solution of the Helmholtz equation with large wave numbers](https://reader035.vdocument.in/reader035/viewer/2022062315/568160df550346895dd00e8c/html5/thumbnails/34.jpg)
July 1, 2010 Parallel solution of the Helmholtz equation 34
![Page 35: Parallel solution of the Helmholtz equation with large wave numbers](https://reader035.vdocument.in/reader035/viewer/2022062315/568160df550346895dd00e8c/html5/thumbnails/35.jpg)
July 1, 2010 Parallel solution of the Helmholtz equation 35
![Page 36: Parallel solution of the Helmholtz equation with large wave numbers](https://reader035.vdocument.in/reader035/viewer/2022062315/568160df550346895dd00e8c/html5/thumbnails/36.jpg)
July 1, 2010 Parallel solution of the Helmholtz equation 36
![Page 37: Parallel solution of the Helmholtz equation with large wave numbers](https://reader035.vdocument.in/reader035/viewer/2022062315/568160df550346895dd00e8c/html5/thumbnails/37.jpg)
July 1, 2010 Parallel solution of the Helmholtz equation 37
![Page 38: Parallel solution of the Helmholtz equation with large wave numbers](https://reader035.vdocument.in/reader035/viewer/2022062315/568160df550346895dd00e8c/html5/thumbnails/38.jpg)
Comparative Timing Results
Methodtime/iter1 proc
time/iter16 proc
it-ratio1 proc
it-ratio16 proc
CARP-CG 0.0978 0.0134 1.00 1.00
Bi-CGSTAB 0.1477 0.0344 1.51 2.56
GMRES(restart=
10)0.1490 0.0212 1.52 1.58
Time/iter of Bi-CGSTAB and GMRES relative to CARP-CGit-ratio = (time/iter of algorithm) / (time/iter of CARP-CG)
Results from CARP-CG paper (PARCO, 2010)
![Page 39: Parallel solution of the Helmholtz equation with large wave numbers](https://reader035.vdocument.in/reader035/viewer/2022062315/568160df550346895dd00e8c/html5/thumbnails/39.jpg)
Timing and Speedup Results
No. proc No. Iter Time (s) Speed-up Efficiency1 7025 1256 1.00 100%2 7039 822 1.53 76.4%4 7066 462 2.72 68.0%8 7115 255 4.92 61.5%15 7206 159 7.92 52.8%
Problem 2, k=600, Ng=8, grid: 763763 582,169 (complex) equations, rel-res<10-7
![Page 40: Parallel solution of the Helmholtz equation with large wave numbers](https://reader035.vdocument.in/reader035/viewer/2022062315/568160df550346895dd00e8c/html5/thumbnails/40.jpg)
SummaryCARP-CG is highly scalable on the
Helmholtz eqn w/high wave numbers Applicable to discontinuous
coefficients Very simple to implement General-purpose – useful also for
other problems with large off-diagonal elements and discontinuous coefficients
![Page 41: Parallel solution of the Helmholtz equation with large wave numbers](https://reader035.vdocument.in/reader035/viewer/2022062315/568160df550346895dd00e8c/html5/thumbnails/41.jpg)
July 1, 2010 Parallel solution of the Helmholtz equation 41
Other Potential ApplicationsOther Potential Applications Fourth-order schemes for the
Helmholtz equation (already have good initial results)
Maxwell equations Saddle-point problems Circuit problems Linear solvers in some eigenvalue
methods ...
![Page 42: Parallel solution of the Helmholtz equation with large wave numbers](https://reader035.vdocument.in/reader035/viewer/2022062315/568160df550346895dd00e8c/html5/thumbnails/42.jpg)
July 1, 2010 Parallel solution of the Helmholtz equation 42
Relevant PublicationsRelevant Publicationshttp://cs.haifa.ac.il/~gordon/pub.htmlCARP: SIAM J Sci Comp 2005CGMN: ACM Trans Math Software 2008Microscopy: J Parallel & Distr Comp 2008Large convection + discontin coef: CMES 2009CARP-CG: Parallel Comp 2010Scaling for discont coef: J Comp & Appl Math 2010
CARP-CG SOFTWARE AVAILABLE ON REQUEST
THANK YOU!