matrix-vector multiplication in sub-quadratic time (some ...the o(n3/log2 n) matrix multiplication...
TRANSCRIPT
![Page 1: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/1.jpg)
Matrix-Vector Multiplication in Sub-Quadratic Time
(Some Preprocessing Required)
Ryan Williams
Carnegie Mellon University
0-0
![Page 2: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/2.jpg)
Introduction
Matrix-Vector Multiplication: Fundamental Operation in Scientific Computing
1
![Page 3: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/3.jpg)
Introduction
Matrix-Vector Multiplication: Fundamental Operation in Scientific Computing
How fast can n × n matrix-vector multiplication be?
1-a
![Page 4: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/4.jpg)
Introduction
Matrix-Vector Multiplication: Fundamental Operation in Scientific Computing
How fast can n × n matrix-vector multiplication be?
Θ(n2) steps just to read the matrix!
1-b
![Page 5: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/5.jpg)
Introduction
Matrix-Vector Multiplication: Fundamental Operation in Scientific Computing
How fast can n × n matrix-vector multiplication be?
Θ(n2) steps just to read the matrix!
Main Result: If we allow O(n2+ε) preprocessing, then matrix-vector
multiplication over any finite semiring can be done in O(n2/(ε log n)2).
1-c
![Page 6: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/6.jpg)
Better Algorithms for Matrix Multiplication
Three of the major developments:
2
![Page 7: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/7.jpg)
Better Algorithms for Matrix Multiplication
Three of the major developments:
• Arlazarov et al., a.k.a. “Four Russians” (1960’s): O(n3/ log n) operations
Uses table lookups
Good for hardware with short vector operations as primitives
2-a
![Page 8: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/8.jpg)
Better Algorithms for Matrix Multiplication
Three of the major developments:
• Arlazarov et al., a.k.a. “Four Russians” (1960’s): O(n3/ log n) operations
Uses table lookups
Good for hardware with short vector operations as primitives
• Strassen (1969): nlog 7
log 2 = O(n2.81) operations
Asymptotically fast, but overhead in the big-O
Experiments in practice are inconclusive about Strassen vs. Four
Russians for Boolean matrix multiplication (Bard, 2006)
2-b
![Page 9: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/9.jpg)
Better Algorithms for Matrix Multiplication
Three of the major developments:
• Arlazarov et al., a.k.a. “Four Russians” (1960’s): O(n3/ log n) operations
Uses table lookups
Good for hardware with short vector operations as primitives
• Strassen (1969): nlog 7
log 2 = O(n2.81) operations
Asymptotically fast, but overhead in the big-O
Experiments in practice are inconclusive about Strassen vs. Four
Russians for Boolean matrix multiplication (Bard, 2006)
• Coppersmith and Winograd (1990): O(n2.376) operations
Not yet practical
2-c
![Page 10: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/10.jpg)
Focus: Combinatorial Matrix Multiplication Algorithms
3
![Page 11: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/11.jpg)
Focus: Combinatorial Matrix Multiplication Algorithms
• Also called non-algebraic; let’s call them non-subtractive
E.g. Four-Russians is combinatorial, Strassen isn’t
3-a
![Page 12: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/12.jpg)
Focus: Combinatorial Matrix Multiplication Algorithms
• Also called non-algebraic; let’s call them non-subtractive
E.g. Four-Russians is combinatorial, Strassen isn’t
More Non-Subtractive Boolean Matrix Mult. Algorithms:
• Atkinson and Santoro: O(n3/ log3/2 n) on a (log n)-word RAM
• Rytter and Basch-Khanna-Motwani: O(n3/ log2 n) on a RAM
• Chan: Four Russians can be implemented on O(n3/ log2 n) on a pointer
machine
3-b
![Page 13: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/13.jpg)
Main Result
The O(n3/ log2 n) matrix multiplication algorithm can be “de-amortized”
4
![Page 14: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/14.jpg)
Main Result
The O(n3/ log2 n) matrix multiplication algorithm can be “de-amortized”
More precisely, we can:
Preprocess an n × n matrix A over a finite semiring in O(n2+ε)
Such that vector multiplications with A can be done in O(n2/(ε log n)2)
4-a
![Page 15: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/15.jpg)
Main Result
The O(n3/ log2 n) matrix multiplication algorithm can be “de-amortized”
More precisely, we can:
Preprocess an n × n matrix A over a finite semiring in O(n2+ε)
Such that vector multiplications with A can be done in O(n2/(ε log n)2)
Allows for “non-subtractive” matrix multiplication to be done on-line
4-b
![Page 16: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/16.jpg)
Main Result
The O(n3/ log2 n) matrix multiplication algorithm can be “de-amortized”
More precisely, we can:
Preprocess an n × n matrix A over a finite semiring in O(n2+ε)
Such that vector multiplications with A can be done in O(n2/(ε log n)2)
Allows for “non-subtractive” matrix multiplication to be done on-line
Can be implemented on a pointer machine
4-c
![Page 17: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/17.jpg)
Main Result
The O(n3/ log2 n) matrix multiplication algorithm can be “de-amortized”
More precisely, we can:
Preprocess an n × n matrix A over a finite semiring in O(n2+ε)
Such that vector multiplications with A can be done in O(n2/(ε log n)2)
Allows for “non-subtractive” matrix multiplication to be done on-line
Can be implemented on a pointer machine
This Talk: The Boolean case
4-d
![Page 18: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/18.jpg)
Preprocessing Phase: The Boolean Case
Partition the input matrix A into blocks of ⌈ε log n⌉ × ⌈ε log n⌉ size:
A1,1
A2,1
A n
ε log n,1
A1,2 A1, n
ε log n
A n
ε log n,
n
ε log n
· · ·
...
· · · · · ·
...
...
Ai,j
ε log n
ε log n
A =
5
![Page 19: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/19.jpg)
Preprocessing Phase: The Boolean Case
Build a graph G with parts P1, . . . , Pn/(ε log n), Q1, . . . , Qn/(ε log n)
2ε log n
P2
...
...
...
...
P nε log n
P1 2ε log n
2ε log n2ε log n
2ε log n 2ε log n
Q1
Q2
Q nε log n
Each part has 2ε log n
vertices, one for each
possible ε log n vector
6
![Page 20: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/20.jpg)
Preprocessing Phase: The Boolean Case
Edges of G: Each vertex v in each Pi has exactly one edge into each Qj
2ε log nPi 2
ε log n Qj
v
Aj,iv
7
![Page 21: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/21.jpg)
Preprocessing Phase: The Boolean Case
Edges of G: Each vertex v in each Pi has exactly one edge into each Qj
2ε log nPi 2
ε log n Qj
v
Aj,iv
Time to build the graph:
n
ε log n·
n
ε log n· 2ε log n
· (ε log n)2 = O(n2+ε)
number
of Qj
number
of Pi
number
of nodes
in Pi
matrix-vector mult
of Aj,i and v
7-a
![Page 22: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/22.jpg)
How to Do Fast Vector Multiplications
Let v be a column vector. Want: A · v.
8
![Page 23: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/23.jpg)
How to Do Fast Vector Multiplications
Let v be a column vector. Want: A · v.
(1) Break up v into ε log n sized chunks:
v =
v1
v2
...
v n
ε log n
8-a
![Page 24: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/24.jpg)
How to Do Fast Vector Multiplications
(2) For each i = 1, . . . , n/(ε log n), look up vi in Pi.
9
![Page 25: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/25.jpg)
How to Do Fast Vector Multiplications
(2) For each i = 1, . . . , n/(ε log n), look up vi in Pi.
2ε log n
P2
...
...
...
...
P nε log n
P1 2ε log n
2ε log n2ε log n
2ε log n 2ε log n
Q1
Q2
Q nε log n
v1
v2
vn/(ε log n)
Takes O(n) time.9-a
![Page 26: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/26.jpg)
How to Do Fast Vector Multiplications
(2) For each i = 1, . . . , n/(ε log n), look up vi in Pi.
2ε log n
P2
...
...
...
...
P nε log n
P1 2ε log n
2ε log n2ε log n
2ε log n 2ε log n
Q1
Q2
Q nε log n
v1
v2
vn/(ε log n)
Takes O(n) time.10
![Page 27: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/27.jpg)
How to Do Fast Vector Multiplications
(3) Look up the neighbors of vi, mark each neighbor found.
11
![Page 28: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/28.jpg)
How to Do Fast Vector Multiplications
(3) Look up the neighbors of vi, mark each neighbor found.
2ε log n
P2
...
...
...
...
P nε log n
P1 2ε log n
2ε log n2ε log n
2ε log n 2ε log n
Q1
Q2
Q nε log n
v1
v2
vn/(ε log n)
A1,1 · v1
A2,1 · v1
A nε log n ,1 · v1
11-a
![Page 29: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/29.jpg)
How to Do Fast Vector Multiplications
(3) Look up the neighbors of vi, mark each neighbor found.
2ε log n
P2
...
...
...
...
P nε log n
P1 2ε log n
2ε log n2ε log n
2ε log n 2ε log n
Q1
Q2
Q nε log n
v1
v2
vn/(ε log n)
A1,2 · v2
A2,2 · v2
A nε log n ,2 · v2
12
![Page 30: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/30.jpg)
How to Do Fast Vector Multiplications
(3) Look up the neighbors of vi, mark each neighbor found.
2ε log n
P2
...
...
...
...
P nε log n
P1 2ε log n
2ε log n2ε log n
2ε log n 2ε log n
Q1
Q2
Q nε log n
v1
v2
vn/(ε log n)
A1, nε log n
· vn/(ε log n)
A2, nε log n
· vn/(ε log n)
A nε log n , n
ε log n· vn/(ε log n)
Takes O
(
(
nε log n
)2)
13
![Page 31: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/31.jpg)
How to Do Fast Vector Multiplications
(4) For each Qj , define v′
j as the OR of all marked vectors in Qj
2ε log n
P2
...
...
...
...
P nε log n
P1 2ε log n
2ε log n2ε log n
2ε log n 2ε log n
Q1
Q2
Q nε log n
v1
v2
vn/(ε log n)
⇒
⇒
⇒
v′1
v′2
v′n/(ε log n)
∨
∨
∨
Takes O(n1+ε) time14
![Page 32: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/32.jpg)
How to Do Fast Vector Multiplications
(4) For each Qj , define v′
j as the OR of all marked vectors in Qj
2ε log n
P2
...
...
...
...
P nε log n
P1 2ε log n
2ε log n2ε log n
2ε log n 2ε log n
Q1
Q2
Q nε log n
v1
v2
vn/(ε log n)
⇒
⇒
⇒
v′1
v′2
v′n/(ε log n)
∨
∨
∨
Takes O(n1+ε) time15
![Page 33: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/33.jpg)
How to Do Fast Vector Multiplications
(5) Output v′ :=
v′
1
v′
2
...
v′n
ε log n
. Claim: v′ = A · v.
16
![Page 34: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/34.jpg)
How to Do Fast Vector Multiplications
(5) Output v′ :=
v′
1
v′
2
...
v′n
ε log n
. Claim: v′ = A · v.
Proof: By definition, v′
j =∨n/(ε log n)
i=1 Aj,i · vi.
16-a
![Page 35: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/35.jpg)
How to Do Fast Vector Multiplications
(5) Output v′ :=
v′
1
v′
2
...
v′n
ε log n
. Claim: v′ = A · v.
Proof: By definition, v′
j =∨n/(ε log n)
i=1 Aj,i · vi.
Av =
A1,1 · · · A1,n/(ε log n)
.... . .
...
An/(ε log n),1 · · · An/(ε log n),n/(ε log n)
v1
...
v n
ε log n
16-b
![Page 36: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/36.jpg)
How to Do Fast Vector Multiplications
(5) Output v′ :=
v′
1
v′
2
...
v′n
ε log n
. Claim: v′ = A · v.
Proof: By definition, v′
j =∨n/(ε log n)
i=1 Aj,i · vi.
Av =
A1,1 · · · A1,n/(ε log n)
.... . .
...
An/(ε log n),1 · · · An/(ε log n),n/(ε log n)
v1
...
v n
ε log n
= (∨n/(ε log n)
i=1 A1,i · vi, . . . ,∨n/(ε log n)
i=1 A1,n/(ε log n) · vi) = v′.
16-c
![Page 37: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/37.jpg)
Some Applications
Can quickly compute the neighbors of arbitrary vertex subsets
Let A be the adjacency matrix of G = (V,E).
Let vS be the indicator vector for a S ⊆ V .
17
![Page 38: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/38.jpg)
Some Applications
Can quickly compute the neighbors of arbitrary vertex subsets
Let A be the adjacency matrix of G = (V,E).
Let vS be the indicator vector for a S ⊆ V .
Proposition: A · vS is the indicator vector for N(S), the neighborhood of S.
17-a
![Page 39: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/39.jpg)
Some Applications
Can quickly compute the neighbors of arbitrary vertex subsets
Let A be the adjacency matrix of G = (V,E).
Let vS be the indicator vector for a S ⊆ V .
Proposition: A · vS is the indicator vector for N(S), the neighborhood of S.
Corollary: After O(n2+ε) preprocessing, can determine the neighborhood of
any vertex subset in O(n2/(ε log n)2) time.
(One level of BFS in o(n2) time)
17-b
![Page 40: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/40.jpg)
Graph Queries
Corollary: After O(n2+ε) preprocessing, can determine if a given vertex
subset is an independent set, a vertex cover, or a dominating set, all in
O(n2/(ε log n)2) time.
18
![Page 41: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/41.jpg)
Graph Queries
Corollary: After O(n2+ε) preprocessing, can determine if a given vertex
subset is an independent set, a vertex cover, or a dominating set, all in
O(n2/(ε log n)2) time.
Proof: Let S ⊆ V .
18-a
![Page 42: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/42.jpg)
Graph Queries
Corollary: After O(n2+ε) preprocessing, can determine if a given vertex
subset is an independent set, a vertex cover, or a dominating set, all in
O(n2/(ε log n)2) time.
Proof: Let S ⊆ V .
S is dominating ⇐⇒ S ∪ N(S) = V .
18-b
![Page 43: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/43.jpg)
Graph Queries
Corollary: After O(n2+ε) preprocessing, can determine if a given vertex
subset is an independent set, a vertex cover, or a dominating set, all in
O(n2/(ε log n)2) time.
Proof: Let S ⊆ V .
S is dominating ⇐⇒ S ∪ N(S) = V .
S is independent ⇐⇒ S ∩ N(S) = ∅.
18-c
![Page 44: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/44.jpg)
Graph Queries
Corollary: After O(n2+ε) preprocessing, can determine if a given vertex
subset is an independent set, a vertex cover, or a dominating set, all in
O(n2/(ε log n)2) time.
Proof: Let S ⊆ V .
S is dominating ⇐⇒ S ∪ N(S) = V .
S is independent ⇐⇒ S ∩ N(S) = ∅.
S is a vertex cover ⇐⇒ V − S is independent.
18-d
![Page 45: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/45.jpg)
Graph Queries
Corollary: After O(n2+ε) preprocessing, can determine if a given vertex
subset is an independent set, a vertex cover, or a dominating set, all in
O(n2/(ε log n)2) time.
Proof: Let S ⊆ V .
S is dominating ⇐⇒ S ∪ N(S) = V .
S is independent ⇐⇒ S ∩ N(S) = ∅.
S is a vertex cover ⇐⇒ V − S is independent.
Each can be quickly determined from knowing S and N(S).
18-e
![Page 46: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/46.jpg)
Triangle Detection
19
![Page 47: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/47.jpg)
Triangle Detection
Problem: Triangle Detection
Given: Graph G and vertex i.
Question: Does i participate in a 3-cycle, a.k.a. triangle?
19-a
![Page 48: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/48.jpg)
Triangle Detection
Problem: Triangle Detection
Given: Graph G and vertex i.
Question: Does i participate in a 3-cycle, a.k.a. triangle?
Worst Case: Can take Θ(n2) time to check all pairs of neighbors of i
19-b
![Page 49: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/49.jpg)
Triangle Detection
Problem: Triangle Detection
Given: Graph G and vertex i.
Question: Does i participate in a 3-cycle, a.k.a. triangle?
Worst Case: Can take Θ(n2) time to check all pairs of neighbors of i
Corollary: After O(n2+ε) preprocessing on G, can solve triangle detection
for arbitrary vertices in O(n2/(ε log n)2) time.
19-c
![Page 50: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/50.jpg)
Triangle Detection
Problem: Triangle Detection
Given: Graph G and vertex i.
Question: Does i participate in a 3-cycle, a.k.a. triangle?
Worst Case: Can take Θ(n2) time to check all pairs of neighbors of i
Corollary: After O(n2+ε) preprocessing on G, can solve triangle detection
for arbitrary vertices in O(n2/(ε log n)2) time.
Proof: Given vertex i, let S be its set of neighbors (gotten in O(n) time).
S is not independent ⇐⇒ i participates in a triangle.
19-d
![Page 51: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/51.jpg)
Conclusion
A preprocessing/multiplication algorithm for matrix-vector multiplication that
builds on lookup table techniques
20
![Page 52: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/52.jpg)
Conclusion
A preprocessing/multiplication algorithm for matrix-vector multiplication that
builds on lookup table techniques
• Is there a preprocessing/multiplication algorithm for sparse matrices? Can
we do multiplication in e.g. O(m/poly(log n) + n),
where m = number of nonzeroes?
20-a
![Page 53: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/53.jpg)
Conclusion
A preprocessing/multiplication algorithm for matrix-vector multiplication that
builds on lookup table techniques
• Is there a preprocessing/multiplication algorithm for sparse matrices? Can
we do multiplication in e.g. O(m/poly(log n) + n),
where m = number of nonzeroes?
• Can the algebraic matrix multiplication algorithms (Strassen, etc.) be
applied to this problem?
20-b
![Page 54: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/54.jpg)
Conclusion
A preprocessing/multiplication algorithm for matrix-vector multiplication that
builds on lookup table techniques
• Is there a preprocessing/multiplication algorithm for sparse matrices? Can
we do multiplication in e.g. O(m/poly(log n) + n),
where m = number of nonzeroes?
• Can the algebraic matrix multiplication algorithms (Strassen, etc.) be
applied to this problem?
• Can our ideas be extended to achieve non-subtractive Boolean matrix
multiplication in o(n3/ log2 n)?
20-c
![Page 55: Matrix-Vector Multiplication in Sub-Quadratic Time (Some ...The O(n3/log2 n) matrix multiplication algorithm can be “de-amortized” More precisely, we can: Preprocess an n×nmatrix](https://reader035.vdocument.in/reader035/viewer/2022071507/6127e7250fae083f09483f50/html5/thumbnails/55.jpg)
Thank you!
21