fill reduction algorithm using diagonal markowitz scheme with local symmetrization patrick amestoy...
TRANSCRIPT
![Page 1: Fill Reduction Algorithm Using Diagonal Markowitz Scheme with Local Symmetrization Patrick Amestoy ENSEEIHT-IRIT, France Xiaoye S. Li Esmond Ng Lawrence](https://reader030.vdocument.in/reader030/viewer/2022032516/56649c785503460f9492d60c/html5/thumbnails/1.jpg)
Fill Reduction Algorithm Using Diagonal Fill Reduction Algorithm Using Diagonal Markowitz Scheme with Local Markowitz Scheme with Local
Symmetrization Symmetrization
Patrick Amestoy
ENSEEIHT-IRIT, France
Xiaoye S. Li
Esmond Ng
Lawrence Berkeley National Laboratory
![Page 2: Fill Reduction Algorithm Using Diagonal Markowitz Scheme with Local Symmetrization Patrick Amestoy ENSEEIHT-IRIT, France Xiaoye S. Li Esmond Ng Lawrence](https://reader030.vdocument.in/reader030/viewer/2022032516/56649c785503460f9492d60c/html5/thumbnails/2.jpg)
SIAM CSE03, Feb 10-13, 2003 2
ContentsContents
Motivation
Graph models for Gaussian elimination
Minimum priority metrics
Experimental results
Summary
Add: Runtime and space complexity
![Page 3: Fill Reduction Algorithm Using Diagonal Markowitz Scheme with Local Symmetrization Patrick Amestoy ENSEEIHT-IRIT, France Xiaoye S. Li Esmond Ng Lawrence](https://reader030.vdocument.in/reader030/viewer/2022032516/56649c785503460f9492d60c/html5/thumbnails/3.jpg)
SIAM CSE03, Feb 10-13, 2003 3
Motivation -- New Sparse LU Factorization AlgorithmsMotivation -- New Sparse LU Factorization Algorithms
Inexpensive pre/post-processing Equilibration (or scaling) Pre-permute rows or columns of A to maximize its diagonal
Find a matching with maximum weight for bipartite graph of A Example: MC64 [Duff/Koster ‘99]
Iterative refinement
GESP (static pivoting) [Li/Demmel ‘98, SuperLU_DIST] Pivots are chosen from the diagonal Allow half-precision perturbation of small diagonals
Unsymmetrized multifrontal [Amestoy/Puglisi ‘00, MA41_NEW] Prefer diagonal pivoting, but threshold pivoting is possible Allow unsymmetric fronts, but dependency graph is still a tree
Diagonal is (almost) goodStruct(L’) Struct(U)
![Page 4: Fill Reduction Algorithm Using Diagonal Markowitz Scheme with Local Symmetrization Patrick Amestoy ENSEEIHT-IRIT, France Xiaoye S. Li Esmond Ng Lawrence](https://reader030.vdocument.in/reader030/viewer/2022032516/56649c785503460f9492d60c/html5/thumbnails/4.jpg)
SIAM CSE03, Feb 10-13, 2003 4
Existing Ordering Strategies to Preserve SparsityExisting Ordering Strategies to Preserve Sparsity
Symmetric ordering algorithms on A’+A Greedy algorithms
e.g., minimum degree, minimum deficiency, etc.
Graph partitioning Hybrid
Problem: unsymmetric structure is not respected!
![Page 5: Fill Reduction Algorithm Using Diagonal Markowitz Scheme with Local Symmetrization Patrick Amestoy ENSEEIHT-IRIT, France Xiaoye S. Li Esmond Ng Lawrence](https://reader030.vdocument.in/reader030/viewer/2022032516/56649c785503460f9492d60c/html5/thumbnails/5.jpg)
SIAM CSE03, Feb 10-13, 2003 5
Structural Gaussian Elimination -- Symmetric CaseStructural Gaussian Elimination -- Symmetric Case
i j k i j k
Eliminate 1
1
i
j
k
1
i
j
k
1
i
j
k
Eliminate 1i
k
j
•Undirected graph•After a vertex is eliminated, all its neighbors become a clique•The edges of the clique are the potential fills (upper bound !)
![Page 6: Fill Reduction Algorithm Using Diagonal Markowitz Scheme with Local Symmetrization Patrick Amestoy ENSEEIHT-IRIT, France Xiaoye S. Li Esmond Ng Lawrence](https://reader030.vdocument.in/reader030/viewer/2022032516/56649c785503460f9492d60c/html5/thumbnails/6.jpg)
SIAM CSE03, Feb 10-13, 2003 6
Structural Gaussian Elimination -- Unsymmetric CaseStructural Gaussian Elimination -- Unsymmetric Case
Eliminate 1
1
r1
r2
c1 c2 c3
1
r1
r2
c1 c2 c3
c1r1
r2c2
c3
Eliminate 1r1
r2
c1
c2
c3
1 1
•Bipartite graph•After a vertex is eliminated, all the row & column vertices adjacent to it become fully connected – “bi-clique” (assuming diagonal pivot)•The edges of the bi-clique are the potential fills (upper bound !)
![Page 7: Fill Reduction Algorithm Using Diagonal Markowitz Scheme with Local Symmetrization Patrick Amestoy ENSEEIHT-IRIT, France Xiaoye S. Li Esmond Ng Lawrence](https://reader030.vdocument.in/reader030/viewer/2022032516/56649c785503460f9492d60c/html5/thumbnails/7.jpg)
SIAM CSE03, Feb 10-13, 2003 7
Ordering Algorithms RevisitOrdering Algorithms Revisit
Markowitz [1957] for unsymmetric matrices At step k, pick pivot in the trailing submatrix so that:
It has minimum , and It is bounded by a numerical threshold
Bound the size of the rank-1 update matrix Expensive to implement because it is mixed with numerical consideration Examples: MA48 (HSL), etc.
“Restricted” Markowitz -- only look ahead a few candidate columns (rows) with the lowest degrees [Zlatev ‘80]
Minimum degree [Tinney/Walker ‘67] Special case of Markowitz for SPD systems Efficient implementation, because:
Diagonal is stable as numerical pivot Use quotient graph as a compact representation without regard of numerical values
ija)1()1( ji cr
![Page 8: Fill Reduction Algorithm Using Diagonal Markowitz Scheme with Local Symmetrization Patrick Amestoy ENSEEIHT-IRIT, France Xiaoye S. Li Esmond Ng Lawrence](https://reader030.vdocument.in/reader030/viewer/2022032516/56649c785503460f9492d60c/html5/thumbnails/8.jpg)
SIAM CSE03, Feb 10-13, 2003 8
Simulation ResultSimulation Result
Order(A) vs. Order(A’+A) (Markowitz vs. min degree) Diagonal pivoting
88 unsymmetric matrices Mean fill ratio 0.90 Mean flops ratio 0.79
54 very unsymmetric (symmetry <= 0.5) Mean fill ratio 0.85 Mean flops ratio 0.56
![Page 9: Fill Reduction Algorithm Using Diagonal Markowitz Scheme with Local Symmetrization Patrick Amestoy ENSEEIHT-IRIT, France Xiaoye S. Li Esmond Ng Lawrence](https://reader030.vdocument.in/reader030/viewer/2022032516/56649c785503460f9492d60c/html5/thumbnails/9.jpg)
SIAM CSE03, Feb 10-13, 2003 9
Quotient Graph – Symmetric CaseQuotient Graph – Symmetric Case
Elements -- representative nodes of the connected components in the
eliminated subgraphVariables -- uneliminated nodes
Current pivot p:
If variable v adjacent to e1, it will be adjacent to p e1 can be absorbed by p p is representative of conn. comp. {e1, e2, p}
e1
e2
px x
x
x
. element list = {e1, e2}
. variable list
v
p p
21 eepp LLAL pA
pA
![Page 10: Fill Reduction Algorithm Using Diagonal Markowitz Scheme with Local Symmetrization Patrick Amestoy ENSEEIHT-IRIT, France Xiaoye S. Li Esmond Ng Lawrence](https://reader030.vdocument.in/reader030/viewer/2022032516/56649c785503460f9492d60c/html5/thumbnails/10.jpg)
SIAM CSE03, Feb 10-13, 2003 10
Quotient Graph -- Unsymmetric CaseQuotient Graph -- Unsymmetric Case
Current pivot p:
p
UUUU
UUUU
LLL
peev
pepe
pee
e2e1path search must
e2or e1 absorbcannot p
,
But
,21
21
,21
Difficulty:Path length may be greater than 2 !
e1
e2
p
x
x
x
v
![Page 11: Fill Reduction Algorithm Using Diagonal Markowitz Scheme with Local Symmetrization Patrick Amestoy ENSEEIHT-IRIT, France Xiaoye S. Li Esmond Ng Lawrence](https://reader030.vdocument.in/reader030/viewer/2022032516/56649c785503460f9492d60c/html5/thumbnails/11.jpg)
SIAM CSE03, Feb 10-13, 2003 11
Quotient Graph -- “Local Symmetrization”Quotient Graph -- “Local Symmetrization”
e1
e2
p
x
x
x
v
Current pivot p:
p} e2, {e1, comp. conn. of tiverepresenta is p
e2 and e1 absorbcan p
21
21
pee
pee
UUU
LLL
Advantage: - Path length bounded by 2 !
Disadvantage: - Lose some asymmetry - More fill
s s
s
![Page 12: Fill Reduction Algorithm Using Diagonal Markowitz Scheme with Local Symmetrization Patrick Amestoy ENSEEIHT-IRIT, France Xiaoye S. Li Esmond Ng Lawrence](https://reader030.vdocument.in/reader030/viewer/2022032516/56649c785503460f9492d60c/html5/thumbnails/12.jpg)
SIAM CSE03, Feb 10-13, 2003 12
Cost of ImplementationCost of Implementation
G(A) viaReachable Set
Quotient Graph Elim. GraphG(L+U)
Symmetric Long search path In-place
Path length 2 In-place [George/Liu ‘81]
Not in-place
Unsym. Long search path In-place [Pagallo/Maulino ‘83]
Local Sym. Path length 2 In-place
Elimination models can be implemented using standard graphs or quotient graphs, with different cost in time & space.
![Page 13: Fill Reduction Algorithm Using Diagonal Markowitz Scheme with Local Symmetrization Patrick Amestoy ENSEEIHT-IRIT, France Xiaoye S. Li Esmond Ng Lawrence](https://reader030.vdocument.in/reader030/viewer/2022032516/56649c785503460f9492d60c/html5/thumbnails/13.jpg)
SIAM CSE03, Feb 10-13, 2003 13
Minimum Priority MetricsMinimum Priority Metrics
Metrics are based on “approximate degree” in the sense of AMD, can be implemented efficiently
Almost the same cost using various metrics: Based on row & column counts:
PRODUCT (a.k.a. Markowitz), SUM, MIN, MAX, etc.
Minimum fill : areas associated with the existing cliques are deducted …...
![Page 14: Fill Reduction Algorithm Using Diagonal Markowitz Scheme with Local Symmetrization Patrick Amestoy ENSEEIHT-IRIT, France Xiaoye S. Li Esmond Ng Lawrence](https://reader030.vdocument.in/reader030/viewer/2022032516/56649c785503460f9492d60c/html5/thumbnails/14.jpg)
SIAM CSE03, Feb 10-13, 2003 14
Preliminary Results with Local SymmetrizationPreliminary Results with Local Symmetrization
Matrices: 98 unsymmetric in structure
Metrics : based on row/column counts or fill
Solvers: MA41_NEW : unsymmetrized multifrontal
Local symmetrization ordering is ideal for this solver SuperLU_DIST : GESP
![Page 15: Fill Reduction Algorithm Using Diagonal Markowitz Scheme with Local Symmetrization Patrick Amestoy ENSEEIHT-IRIT, France Xiaoye S. Li Esmond Ng Lawrence](https://reader030.vdocument.in/reader030/viewer/2022032516/56649c785503460f9492d60c/html5/thumbnails/15.jpg)
SIAM CSE03, Feb 10-13, 2003 15
Compare Different MetricsCompare Different Metrics
Solver: MA41_NEWAverage fill ratio using various metrics with respect to Markowitz
(product of row & col counts)
Metrics Mean fill ratio
SUM row & col counts 0.999
MAX row & col counts 6.079
MIN row & col counts 15.94
Approx. min fill (AMF1) 0.965
Approx. min fill (AMF4) 0.959
![Page 16: Fill Reduction Algorithm Using Diagonal Markowitz Scheme with Local Symmetrization Patrick Amestoy ENSEEIHT-IRIT, France Xiaoye S. Li Esmond Ng Lawrence](https://reader030.vdocument.in/reader030/viewer/2022032516/56649c785503460f9492d60c/html5/thumbnails/16.jpg)
SIAM CSE03, Feb 10-13, 2003 16
Compare with AMD(A’+A) using Min Fill -- All Compare with AMD(A’+A) using Min Fill -- All UnsymmetricUnsymmetric
MA41_NEW
SuperLU_DIST
Fill ratio Flops ratio
Mean 0.96 0.92
Best / worst 0.41 / 1.27 0.13 / 2.38
Fill ratio Flops ratio
Mean 0.96 0.96
Best / worst 0.38 / 2.36 0.009 / 6.00
![Page 17: Fill Reduction Algorithm Using Diagonal Markowitz Scheme with Local Symmetrization Patrick Amestoy ENSEEIHT-IRIT, France Xiaoye S. Li Esmond Ng Lawrence](https://reader030.vdocument.in/reader030/viewer/2022032516/56649c785503460f9492d60c/html5/thumbnails/17.jpg)
SIAM CSE03, Feb 10-13, 2003 17
Compare with AMD(A’+A) using Min Fill -- Very Compare with AMD(A’+A) using Min Fill -- Very UnsymmetricUnsymmetric
MA41_NEW
SuperLU_DIST
Fill Flops
Mean 0.88 0.77
Best / worst 0.38 / 1.18 0.009 / 1.69
Fill Flops
Mean 0.95 0.89
Best / worst 0.41 / 1.27 0.13 / 2.38
![Page 18: Fill Reduction Algorithm Using Diagonal Markowitz Scheme with Local Symmetrization Patrick Amestoy ENSEEIHT-IRIT, France Xiaoye S. Li Esmond Ng Lawrence](https://reader030.vdocument.in/reader030/viewer/2022032516/56649c785503460f9492d60c/html5/thumbnails/18.jpg)
SIAM CSE03, Feb 10-13, 2003 18
SummarySummary
First implementation based on BQG model Features: supervariable, element absorption, mass elimination
Using approximate degree (degree upper bound)Tried various metrics on large collection of matrices
PRODUCT, SUM, MIN-FILL, etc. Not a single one is universally best, MIN-FILL is often better
Local symmetrization Cheaper to implement, harder to understand behavior Especially suitable for unsymmetrized multifrontal, also benefit GESP Respectable gain for very unsymmetric matrices
![Page 19: Fill Reduction Algorithm Using Diagonal Markowitz Scheme with Local Symmetrization Patrick Amestoy ENSEEIHT-IRIT, France Xiaoye S. Li Esmond Ng Lawrence](https://reader030.vdocument.in/reader030/viewer/2022032516/56649c785503460f9492d60c/html5/thumbnails/19.jpg)
SIAM CSE03, Feb 10-13, 2003 19
Summary (con’d)Summary (con’d)
Results for very unsymmetric matrices
Future work Work underway for a fully unsymmetric version Extend to graph partitioning strategy
Local Sym. Unsym. (simulation)
Fill reduction 0.88 0.85
Flops reduction 0.77 0.56
![Page 20: Fill Reduction Algorithm Using Diagonal Markowitz Scheme with Local Symmetrization Patrick Amestoy ENSEEIHT-IRIT, France Xiaoye S. Li Esmond Ng Lawrence](https://reader030.vdocument.in/reader030/viewer/2022032516/56649c785503460f9492d60c/html5/thumbnails/20.jpg)
SIAM CSE03, Feb 10-13, 2003 20
The EndThe End
![Page 21: Fill Reduction Algorithm Using Diagonal Markowitz Scheme with Local Symmetrization Patrick Amestoy ENSEEIHT-IRIT, France Xiaoye S. Li Esmond Ng Lawrence](https://reader030.vdocument.in/reader030/viewer/2022032516/56649c785503460f9492d60c/html5/thumbnails/21.jpg)
SIAM CSE03, Feb 10-13, 2003 21
1 x 2 xx x 3 x 4 x 5 x x x 6 x x 7
ExampleExample
2
3
4
5
7
6
1
2
3
4
5
7
6
1A
G(A)
row column