communication complexity lower bound techniques for...
TRANSCRIPT
![Page 1: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/1.jpg)
1-1
Lower Bound Techniques for MultipartyCommunication Complexity
Qin Zhang
Indiana University Bloomington
Based on works with Jeff Phillips, Elad Verbin and David Woodruff
![Page 2: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/2.jpg)
2-1
The multiparty number-in-hand (NIH) comm.
x1 = 010011 x2 = 111011
x3 = 111111
xk = 100011
They want to jointly compute f(x1, x2, . . . , xk)
Goal: minimize total bits of communication
– A model for proving lower bounds
![Page 3: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/3.jpg)
2-2
The multiparty number-in-hand (NIH) comm.
x1 = 010011 x2 = 111011
x3 = 111111
xk = 100011
They want to jointly compute f(x1, x2, . . . , xk)
Message-Passing (MP):If x1 talks to x2, otherscannot hear.
Blackboard (BB): Onespeaks, everyone hears.
Goal: minimize total bits of communication
– A model for proving lower bounds
![Page 4: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/4.jpg)
2-3
The multiparty number-in-hand (NIH) comm.
x1 = 010011 x2 = 111011
x3 = 111111
xk = 100011
They want to jointly compute f(x1, x2, . . . , xk)
Message-Passing (MP):If x1 talks to x2, otherscannot hear.
Blackboard (BB): Onespeaks, everyone hears.
· · ·S1 S2 S3 Sk
C=Coordinator
Goal: minimize total bits of communication
– A model for proving lower bounds
![Page 5: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/5.jpg)
3-1
Previously, for multiparty NIH comm. model
Well studied? YES and NO.
![Page 6: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/6.jpg)
3-2
Previously, for multiparty NIH comm. model
Well studied? YES and NO.
Quite a few papers in BB model:
[Alon et al. ’96], [Bar-Yossef et al. ’04], . . .
![Page 7: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/7.jpg)
3-3
Previously, for multiparty NIH comm. model
Well studied? YES and NO.
Quite a few papers in BB model:
But, Almost nothing in MP model before 2012.
[Alon et al. ’96], [Bar-Yossef et al. ’04], . . .
![Page 8: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/8.jpg)
3-4
Previously, for multiparty NIH comm. model
Well studied? YES and NO.
Quite a few papers in BB model:
But, Almost nothing in MP model before 2012.
• Too easy?
[Alon et al. ’96], [Bar-Yossef et al. ’04], . . .
![Page 9: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/9.jpg)
3-5
Previously, for multiparty NIH comm. model
Well studied? YES and NO.
Quite a few papers in BB model:
But, Almost nothing in MP model before 2012.
• Too easy?
• Lack of motivations?
[Alon et al. ’96], [Bar-Yossef et al. ’04], . . .
![Page 10: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/10.jpg)
4-1
Too easy?
Could be more difficult than BB, since in MP each playerobserves only a piece of protocol transcript.
![Page 11: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/11.jpg)
4-2
Too easy?
Could be more difficult than BB, since in MP each playerobserves only a piece of protocol transcript.
We do not want to analyze a k-dimensional matrix directly.
0 1
0
1
0 0
01
AB A communication matrix of the
2-party AND function
![Page 12: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/12.jpg)
4-3
Too easy?
Could be more difficult than BB, since in MP each playerobserves only a piece of protocol transcript.
We do not want to analyze a k-dimensional matrix directly.
0 1
0
1
0 0
01
AB A communication matrix of the
2-party AND function
Even for the 2-party Gap-Hamming problem, where Alicehas a n-bit vector and Bob has a n-bit vector, and they wantto compute if the hamming distance of these two vectorsis bigger than n/2 +
√n or less than n/2 −
√n, it tooks
researchers in this area about 10 years and dozens of papers(IW03, Woodruff04, . . ., Sherstov12) to fully understand its2-dim communication matrix!
![Page 13: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/13.jpg)
5-1
Lack of motivations?
(Continous) Distributed Monitoringcommunication → energy, bandwidth, . . .
Data in the cloud
Sensor networksNetwork routers
![Page 14: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/14.jpg)
6-1
Lack of motivations? (cont.)
Data Streamscommunication → space
Memory
CPU
![Page 15: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/15.jpg)
6-2
Lack of motivations? (cont.)
Data Streamscommunication → space
Memory
CPUP1 P2 Pk
![Page 16: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/16.jpg)
7-1
The BSP model.E.g., Apache Giraph, Pregel.
Input
MapShuffle
Reduce
Output
The MapReduce model.E.g., Hadoop, Google MapReduce.
Distributed Computation for Big Datacommunication → bandwidth, . . .
Lack of motivations? (cont.)
![Page 17: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/17.jpg)
8-1
Interesting problems
• Frequency moments Fp
F0: #distinct elements
F2: size of self-join
• Heavy hitters
• Quantile
• Entropy
• . . .
Statisticalproblems
![Page 18: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/18.jpg)
8-2
Interesting problems
• Frequency moments Fp
F0: #distinct elements
F2: size of self-join
• Heavy hitters
• Quantile
• Entropy
• . . .
Statisticalproblems Graph problems
• Connectivity
• Bipartiteness
• Counting triangles
• Matching
• Minimum spanning tree
• . . .
![Page 19: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/19.jpg)
8-3
Interesting problems
• Frequency moments Fp
F0: #distinct elements
F2: size of self-join
• Heavy hitters
• Quantile
• Entropy
• . . .
Statisticalproblems Graph problems
• Connectivity
• Bipartiteness
• Counting triangles
• Matching
• Minimum spanning tree
• . . .
Numericallinear algebra
• Lp regression
• Low-rank
approximation
• . . .
![Page 20: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/20.jpg)
8-4
Interesting problems
• Frequency moments Fp
F0: #distinct elements
F2: size of self-join
• Heavy hitters
• Quantile
• Entropy
• . . .
Statisticalproblems Graph problems
• Connectivity
• Bipartiteness
• Counting triangles
• Matching
• Minimum spanning tree
• . . .
Numericallinear algebra
• Lp regression
• Low-rank
approximation
• . . .
DB queries
• Conjuntive
queries
Distributedlearning
• Classifiers
. . .
Geometryproblems
![Page 21: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/21.jpg)
9-1
This talk
Introduce two general techniques thatwork in both BB model and MP modelfor proving lower bounds, with examples.
Conclude with several future directions.
We will talk lower bounds.Most lower bounds match the upper bounds.
![Page 22: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/22.jpg)
10-1
Our new ideas and techniques
Ideas
1. Reduce a k-player problem to a 2-player problem.
2. Decompose a complicated problem to several simpler problems
![Page 23: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/23.jpg)
10-2
Our new ideas and techniques
Ideas
1. Reduce a k-player problem to a 2-player problem.
2. Decompose a complicated problem to several simpler problems
In fact, we are
exploring the underlying patterns of communication matrices
![Page 24: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/24.jpg)
10-3
Our new ideas and techniques
Ideas
1. Reduce a k-player problem to a 2-player problem.
2. Decompose a complicated problem to several simpler problems
In fact, we are
exploring the underlying patterns of communication matrices
SymmetrizationWith Phillips and Verbin, 2012
CompositionWith Woodruff, 2012
1.
2.
Concrete techniques
![Page 25: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/25.jpg)
10-4
Our new ideas and techniques
Ideas
1. Reduce a k-player problem to a 2-player problem.
2. Decompose a complicated problem to several simpler problems
In fact, we are
exploring the underlying patterns of communication matrices
SymmetrizationWith Phillips and Verbin, 2012
CompositionWith Woodruff, 2012
1.
2.
Concrete techniques
Example: Graph Connecitivy
Example: Distinct Elements
![Page 26: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/26.jpg)
11-1
Communication complexity
I choose not to introduce formal definitions/concepts ofcommunication complexity. Just want to mentionYao’s lemma:
We will prove lower bounds for randomized protocols. Todo that we will usually pick some input distribution, andthen prove lower bounds for any deterministic protocol thatsucceeds w.pr 0.99 under that input distribution.
![Page 27: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/27.jpg)
12-1
1st Technique: Symmetrization
![Page 28: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/28.jpg)
13-1
Example: graph connectivity
k sites each holds a set of edges of a graph
Goal: compute whether the graph is connected.
A trivial UB: O(nk log n) bits.
We prove that this is tight up to a log factor.
![Page 29: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/29.jpg)
14-1
Set disjointness (2-DISJ)
X ⊆ 1, . . . , n Y ⊆ 1, . . . , n
X ∩ Y = ∅?
![Page 30: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/30.jpg)
14-2
Set disjointness (2-DISJ)
X ⊆ 1, . . . , n Y ⊆ 1, . . . , n
X ∩ Y = ∅?
Exists a hard distribution µβ , under which
|X ∩ Y | = 1 (YES instance) w.p. β and
|X ∩ Y | = 0 (NO instance) w.p. 1− β.
Lemma: (Generalization of [Razborov ’90, BJKS ’04])
E[CCβ/100µβ(2-DISJ)] = Ω(n)
![Page 31: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/31.jpg)
15-1
2-DISJ ⇒ k-CONNx1
x2
x3
x4
x5Alice and Bob want to solve the 2-DISJ(X,Y ) ∼ µβ . (set β = 1/k2)⇒ running a protocol for k-CONN on:
Alice
Bob
Alice plays a random site SI with input XI = X.Bob plays other k − 1 sites, and constructs inputs for them using Y : itchoose Xi ∼ µβ |Y (i 6= I).Note: X1, . . . , Xk are i.i.d. conditioned on Y .
Vertices u1, . . . , uk, v1, . . . , vn. An edge (ui, vj) exists iff Xi,j = 1.Let ν be the distribution of the graph.
![Page 32: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/32.jpg)
16-1
2-DISJ ⇒ k-CONN (cont.)
vj |j ∈ [r]\Y vj |j ∈ Y
vj|Y |+1vj|Y |+2
vj|Y |+3vjn vj1 vj|Y |vj2
u1 u3u2 uk
(ui, vj) exists if and only if Xi,j = 1
XI ∩ Y = ∅?
![Page 33: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/33.jpg)
17-1
2-DISJ ⇒ k-CONNx1
x2
x3
x4
x5Alice and Bob want to solve the 2-DISJ(X,Y ) ∼ µβ . (set β = 1/k2)⇒ running a protocol for k-CONN on:
Alice
Bob
Alice plays a random site SI with input XI = X.Bob plays other k − 1 sites, and constructs inputs for them using Y : itchoose Xi ∼ µβ |Y (i 6= I).Note: X1, . . . , Xk are i.i.d. conditioned on Y .
Correctness: Solving k-CONN w.pr. 1 − β1.5 under ν also solves2-DISJ w.pr. 1− β/100 under µβ .
Vertices u1, . . . , uk, v1, . . . , vn. An edge (ui, vj) exists iff Xi,j = 1.Let ν be the distribution of the graph.
![Page 34: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/34.jpg)
17-2
2-DISJ ⇒ k-CONNx1
x2
x3
x4
x5Alice and Bob want to solve the 2-DISJ(X,Y ) ∼ µβ . (set β = 1/k2)⇒ running a protocol for k-CONN on:
Alice
Bob
Alice plays a random site SI with input XI = X.Bob plays other k − 1 sites, and constructs inputs for them using Y : itchoose Xi ∼ µβ |Y (i 6= I).Note: X1, . . . , Xk are i.i.d. conditioned on Y .
Correctness: Solving k-CONN w.pr. 1 − β1.5 under ν also solves2-DISJ w.pr. 1− β/100 under µβ .
Vertices u1, . . . , uk, v1, . . . , vn. An edge (ui, vj) exists iff Xi,j = 1.Let ν be the distribution of the graph.
E[CCβ/100µβ (2-DISJ)] = O
(1k
)· CCβ
1.5
ν (k-CONN)
![Page 35: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/35.jpg)
17-3
2-DISJ ⇒ k-CONNx1
x2
x3
x4
x5Alice and Bob want to solve the 2-DISJ(X,Y ) ∼ µβ . (set β = 1/k2)⇒ running a protocol for k-CONN on:
Alice
Bob
Alice plays a random site SI with input XI = X.Bob plays other k − 1 sites, and constructs inputs for them using Y : itchoose Xi ∼ µβ |Y (i 6= I).Note: X1, . . . , Xk are i.i.d. conditioned on Y .
Correctness: Solving k-CONN w.pr. 1 − β1.5 under ν also solves2-DISJ w.pr. 1− β/100 under µβ .
Vertices u1, . . . , uk, v1, . . . , vn. An edge (ui, vj) exists iff Xi,j = 1.Let ν be the distribution of the graph.
E[CCβ/100µβ (2-DISJ)] = O
(1k
)· CCβ
1.5
ν (k-CONN)
Ω(n)
![Page 36: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/36.jpg)
17-4
2-DISJ ⇒ k-CONNx1
x2
x3
x4
x5Alice and Bob want to solve the 2-DISJ(X,Y ) ∼ µβ . (set β = 1/k2)⇒ running a protocol for k-CONN on:
Alice
Bob
Alice plays a random site SI with input XI = X.Bob plays other k − 1 sites, and constructs inputs for them using Y : itchoose Xi ∼ µβ |Y (i 6= I).Note: X1, . . . , Xk are i.i.d. conditioned on Y .
Correctness: Solving k-CONN w.pr. 1 − β1.5 under ν also solves2-DISJ w.pr. 1− β/100 under µβ .
Vertices u1, . . . , uk, v1, . . . , vn. An edge (ui, vj) exists iff Xi,j = 1.Let ν be the distribution of the graph.
E[CCβ/100µβ (2-DISJ)] = O
(1k
)· CCβ
1.5
ν (k-CONN)
Ω(n) Ω(nk)
![Page 37: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/37.jpg)
18-1
A useful message
We can show trivial UBs are tight for many statisticaland graph problems in the MP model (e.g., #distinctelements, bipartiteness, cycle/triangle-freeness, . . .), whenexact answers are wanted. (Woodruff, Z, 2013a)
Approximation is often necessary if we want comm.efficient protocols.
![Page 38: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/38.jpg)
18-2
A useful message
We can show trivial UBs are tight for many statisticaland graph problems in the MP model (e.g., #distinctelements, bipartiteness, cycle/triangle-freeness, . . .), whenexact answers are wanted. (Woodruff, Z, 2013a)
Approximation is often necessary if we want comm.efficient protocols.
Example:
Computing the diameter of a graph exactly needs Ω(kn2) bits(when m = n2) in the MP model.
However, there is a protocol that computes the diameter up to
an additive error of 2 using O(kn3/2) bits.
![Page 39: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/39.jpg)
19-1
Other problems using symmetrization
• k-bitwise-MAJ
• k-bitwise-OR/AND/XOR
With applications in
– Distinct elements reporting
– ε-kernels (geometry)
– Heavy-hitters
• Testing bipartiteness,
Testing cycle
Testing triangle-freeness,
. . .
![Page 40: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/40.jpg)
19-2
Other problems using symmetrization
• k-bitwise-MAJ
• k-bitwise-OR/AND/XOR
With applications in
– Distinct elements reporting
– ε-kernels (geometry)
– Heavy-hitters
• Testing bipartiteness,
Testing cycle
Testing triangle-freeness,
. . .
Key:
Find the right 2-playerproblem for reduction
![Page 41: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/41.jpg)
20-1
2nd Technique: Composition
![Page 42: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/42.jpg)
21-1
Example – the F0 (distinct elements) problem
k sites each holds a set Xi (i ∈ 1, 2, . . . , k).
Goal: compute #distinct elements(∪ki=1Xi) up to a (1 + ε)-approx.
![Page 43: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/43.jpg)
21-2
Example – the F0 (distinct elements) problem
k sites each holds a set Xi (i ∈ 1, 2, . . . , k).
Goal: compute #distinct elements(∪ki=1Xi) up to a (1 + ε)-approx.
Current best UB: O(k/ε2)[Cormode, Muth, Yi. 2008]
Holds in MP model
![Page 44: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/44.jpg)
21-3
Example – the F0 (distinct elements) problem
k sites each holds a set Xi (i ∈ 1, 2, . . . , k).
Goal: compute #distinct elements(∪ki=1Xi) up to a (1 + ε)-approx.
Current best UB: O(k/ε2)[Cormode, Muth, Yi. 2008]
Holds in MP model
Previous LB: Ω(k)
And Ω(1/ε2)Direct reduction from Gap-Hamming
![Page 45: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/45.jpg)
21-4
Example – the F0 (distinct elements) problem
k sites each holds a set Xi (i ∈ 1, 2, . . . , k).
Goal: compute #distinct elements(∪ki=1Xi) up to a (1 + ε)-approx.
Current best UB: O(k/ε2)[Cormode, Muth, Yi. 2008]
Holds in MP model
Previous LB: Ω(k)
And Ω(1/ε2)Direct reduction from Gap-Hamming
New LB: Ω(k/ε2). [Woodruff, Z. 2012, 2013b]. Holdsin MP model. Better UB in the BB model
![Page 46: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/46.jpg)
22-1
The proof framework
Step 1: Find two modular problems k-GAP-MAJ and 2-DISJof simpler structures s.t.
F0 can be thought as a composition of them.
Step 2: Analyze the complexities of k-GAP-MAJ and 2-DISJ.
Step 3: Compose two modular problems so that:
Complexity(F0) = Complexity(k-GAP-MAJ)
× Complexity(2-DISJ).
![Page 47: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/47.jpg)
22-2
The proof framework
Step 1: Find two modular problems k-GAP-MAJ and 2-DISJof simpler structures s.t.
F0 can be thought as a composition of them.
Step 2: Analyze the complexities of k-GAP-MAJ and 2-DISJ.
Step 3: Compose two modular problems so that:
Complexity(F0) = Complexity(k-GAP-MAJ)
× Complexity(2-DISJ).
The proof presented here is for a special case when k = 2/ε2.
![Page 48: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/48.jpg)
23-1
k-GAP-MAJ
k sites each holds a bit Zi chosen uniform at random from 0, 1
Goal: compute
k-GAP-MAJ(Z1, Z2, . . . , Zk) =
0, if
∑i∈[k] Zi ≤ k/2−
√k,
1, if∑i∈[k] Zi ≥ k/2 +
√k,
don’t care, otherwise,
![Page 49: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/49.jpg)
23-2
k-GAP-MAJ
k sites each holds a bit Zi chosen uniform at random from 0, 1
Goal: compute
k-GAP-MAJ(Z1, Z2, . . . , Zk) =
0, if
∑i∈[k] Zi ≤ k/2−
√k,
1, if∑i∈[k] Zi ≥ k/2 +
√k,
don’t care, otherwise,
Lemma: Any protocol Π that computes k-GAP-MAJ correctlyw.p. 0.9999 has to learn Ω(k) Zi’s well, that is,
H(Zi | Π) ≤ Hb(1/100) for Ω(k) of i.
In other words: I(Π;Z1, . . . , Zk) = Ω(k).
![Page 50: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/50.jpg)
24-1
Set disjointness (2-DISJ)
X ⊆ 1, . . . , n Y ⊆ 1, . . . , n
X ∩ Y = ∅?
![Page 51: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/51.jpg)
24-2
Set disjointness (2-DISJ)
X ⊆ 1, . . . , n Y ⊆ 1, . . . , n
X ∩ Y = ∅?
Exists a hard distribution µβ , under which
|X ∩ Y | = 1 (YES instance) w.p. β and
|X ∩ Y | = 0 (NO instance) w.p. 1− β.
Lemma: (Generalization of [Razborov ’90, BJKS ’04])
E[CCβ/100µβ(2-DISJ)] = Ω(n)
![Page 52: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/52.jpg)
25-1
The proof sketch
· · ·S1 S2 S3 Sk
Ccoordinator
sites
Y
X1 X2 X3 Xk
2-DISJ
(Xi, Y ) ∼ µ1/2 (hard input dist. for DISJ) for each i ∈ [k]
![Page 53: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/53.jpg)
25-2
The proof sketch
· · ·S1 S2 S3 Sk
Ccoordinator
sites
Y
X1 X2 X3 Xk
2-DISJ
(Xi, Y ) ∼ µ1/2 (hard input dist. for DISJ) for each i ∈ [k]
Zi = |Xi ∩ Y |
1 w.p. 1/20 w.p. 1/2
![Page 54: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/54.jpg)
25-3
The proof sketch
· · ·S1 S2 S3 Sk
Ccoordinator
sites
Y
X1 X2 X3 Xk
2-DISJ
F0(X1, X2, . . . , Xk) ⇐⇒ k-GAP-MAJ(Z1, Z2, . . . , Zk)
(Zi = |Xi ∩ Y |)=⇒ learn Ω(k) Zi’s well
=⇒ need Ω(nk) bits
(using an embedding argument)
(Xi, Y ) ∼ µ1/2 (hard input dist. for DISJ) for each i ∈ [k]
Zi = |Xi ∩ Y |
1 w.p. 1/20 w.p. 1/2
![Page 55: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/55.jpg)
25-4
The proof sketch
· · ·S1 S2 S3 Sk
Ccoordinator
sites
Y
X1 X2 X3 Xk
Q.E.D.
For the reduction setuniverse n = Θ(1/ε2)
we get Ω(k/ε2) LB.
2-DISJ
F0(X1, X2, . . . , Xk) ⇐⇒ k-GAP-MAJ(Z1, Z2, . . . , Zk)
(Zi = |Xi ∩ Y |)=⇒ learn Ω(k) Zi’s well
=⇒ need Ω(nk) bits
(using an embedding argument)
(Xi, Y ) ∼ µ1/2 (hard input dist. for DISJ) for each i ∈ [k]
Zi = |Xi ∩ Y |
1 w.p. 1/20 w.p. 1/2
![Page 56: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/56.jpg)
26-1
Other problems using composition
• Frequency moments Fp (p > 1)
• Entropy
• `p
• Heavy-hitters
• Quantile
![Page 57: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/57.jpg)
26-2
Other problems using composition
• Frequency moments Fp (p > 1)
• Entropy
• `p
• Heavy-hitters
• Quantile
Key:
Find the right modular problems
![Page 58: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/58.jpg)
27-1
Furture directions
![Page 59: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/59.jpg)
28-1
Future directions – Problem space
The current understandings
Exactcomputation
Approximation
statistics graph linearalgebra
DBqueries
well
well
medium
limited
limited
limitedlimited
![Page 60: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/60.jpg)
28-2
Future directions – Problem space
Concretely, e.g.,
What is the CC of computing a const approx of the size ofthe matching?
The current understandings
Exactcomputation
Approximation
statistics graph linearalgebra
DBqueries
well
well
medium
limited
limited
limitedlimited
![Page 61: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/61.jpg)
29-1
Future directions – Problem space (cont.)
Communication complexity of distributed property testing.
Communication complexity of distributed property testing canalso be used for proving LBs for traditional property testing,by extending a framework by Blais et al. 2011.
![Page 62: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/62.jpg)
29-2
Future directions – Problem space (cont.)
Communication complexity of distributed property testing.
Communication complexity of distributed property testing canalso be used for proving LBs for traditional property testing,by extending a framework by Blais et al. 2011.
Communication complexity of relations(instead of functions, useful for database queries)
![Page 63: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/63.jpg)
29-3
Future directions – Problem space (cont.)
Communication complexity of distributed property testing.
Communication complexity of distributed property testing canalso be used for proving LBs for traditional property testing,by extending a framework by Blais et al. 2011.
Communication complexity of relations(instead of functions, useful for database queries)
Need to understand more primitive problems (building blocks),e.g., breadth-first search (BFS), pointer-chasing, etc.
So far building blocks are quite limited: mainly 2-DISJ, 2-GAP-HAMMING, k-GAP-MAJ, k-OR and their variants.
![Page 64: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/64.jpg)
30-1
Future directions – Model/Technique space
Can we relax or generalize the symmetrization technique?
E.g., do not require “totally symmetric distributions”
![Page 65: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/65.jpg)
30-2
Future directions – Model/Technique space
Can we relax or generalize the symmetrization technique?
E.g., do not require “totally symmetric distributions”
Consider round complexity (practical needs).
Consider players with limited space (practical needs).
![Page 66: Communication Complexity Lower Bound Techniques for …homes.sice.indiana.edu/qzhangcs/papers/Multi-CC-distribute.pdf2-1 The multiparty number-in-hand (NIH) comm. x 1 = 010011 x 2](https://reader034.vdocument.in/reader034/viewer/2022050110/5f47e90571147d34025fc70a/html5/thumbnails/66.jpg)
31-1
The end
T HANK YOU
Questions?