tight bounds for distributed functional monitoring david woodruff ibm almaden qin zhang aarhus...
TRANSCRIPT
Tight Bounds for Distributed Functional Monitoring
David Woodruff
IBM Almaden
Qin Zhang
Aarhus University
MADALGO
Based on a paper in STOC, 2012
k-party Number-In-Hand ModelP1
P2
P3
Pk
P4
…
x1
x2
x3
x4
xk
Goals: - compute a function f(x1, …, xk)- minimize communication complexity
-Player to player communication - Protocol transcript always determines who speaks next
-Player to player communication - Protocol transcript always determines who speaks next
k-party Number-In-Hand Model
C
P1 P2 P3 Pk…
Convenient to introduce a “coordinator” C
All communication goes through the coordinator
Communication only affected by a factor of 2
x1 x2 x3 xk
Model Motivation• Data distributed and stored in the cloud
– Impractical to put data on a single device
• Sensor networks– Communication is power-intensive
• Network routers– Bandwidth limitations
• Distributed functional monitoringAuthors: Can, Cormode, Huang, Muthukrishnan, Patt-
Shamir, Shafrir, Tirthapura, Wang, Yi, Zhao, …
k-Party Number-In-Hand Model
C
P1 P2 P3 Pk…
x1 x2 x3 xk
Which functions do we care about? - 8i, xi 2 {0,1, … n}n
- x = x1 + x2 + … + xk
- f(x) = |x|p = (Σi xip)1/p
- |x|0 is number of non-zero coordinates - Talk will focus on |x|0 and |x|2
For distributed databases:
|x|0 is number of distinct elements
|x|22 is known as self-join size
|x|2 useful for regression, low-rank approx
For distributed databases:
|x|0 is number of distinct elements
|x|22 is known as self-join size
|x|2 useful for regression, low-rank approx
Important for applications that the xi are non-
negative
Important for applications that the xi are non-
negative
Randomized Communication Complexity
• What is the randomized communication cost of f?• i.e., the minimal cost of a protocol, which for every set of
inputs, fails in computing f with probability < 1/3
• (n) cost for |x|0 and |x|2
• Reduction from 2-Player Set-Disjointness (DISJ)• Alice has a set S µ [n]• Bob has a set T µ [n] • Either |S Å T| = 0 or |S Å T| = 1• |S Å T| = 1 ! DISJ(S,T) = 1, |S Å T| = 0 !DISJ(S,T) = 0• [KS, R] (n) communication
• Prohibitive
Approximate Answers
Compute a relation with probability > 2/3:
f(x)2(1 ± ε) |x|0
f(x)2(1 ± ε) |x|2
What is the randomized communication cost as a function of k, ε, and n?
Will ignore log(nk/ε) factors
Understanding dependence on ε is critical, e.g., ε<.01
Previous Results
• |x|0: (k + ε-2) and O(k¢ε-2 )
• |x|2: (k + ε-2) and O(k¢ε-2 )
Our Results
• |x|0: (k + ε-2) and O(k¢ε-2 ) (k¢ε-2)
• |x|2: (k + ε-2) and O(k¢ε-2 ) (k¢ε-2)
First lower bounds to depend on
product of k and ε-
2
First lower bounds to depend on
product of k and ε-
2
Implications for data streams:- First tight space lower bound for estimating number of distinct elements without using the Gap-Hamming Problem
- Improves lower bound for estimation of |x|p, p > 2
Previous Lower Bounds• Lower bounds for |x|0 and |x|
• [CMY](k)
• [ABC] (ε-2) • Reduction from Gap-Orthogonality (GAP-ORT)
• P1, P2 have u, v 2 {0,1}ε-2 , respectively
• |¢(u, v) – 1/(2ε2)| < 1/ε or |¢(u, v) - 1/(2ε2)| > 2/ε
• [CR, S] (ε-2) communication
Talk Outline
• Lower Bounds– |x|0
– |x|2
Lower Bound for |x|0
• Improve bound to optimal (k¢ε-2)
• Study a simpler problem: k-GAP-THRESH
– Each player Pi holds a bit Zi
– Zi are i.i.d. Bernoulli(¯)
– Decide if
i=1k Zi > ¯ k + (¯ k)1/2 or i=1
k Zi < ¯ k - (¯ k)1/2
Otherwise don’t care
• Rectangle property: for any correct protocol transcript ¿,
Z1, Z2, …, Zk are independent conditioned on ¿
Rectangle Property of Communication
• Let r be the randomness of C, P1, …, Pk
• For any fixed r, the set S of inputs giving rise to a transcript ¿ is a combinatorial rectangle: S = S1 x S2 x … x Sk
• If input distribution is a product distribution, conditioned on ¿ and r, inputs are independent
• Since this holds for every r, inputs are independent conditioned on ¿
k-GAP-THRESH
C
P1 P2 P3 Pk…
Z1 Z2 Z3 Zk
• The Zi are i.i.d. Bernoulli(¯) • Coordinator wants to decide if: i=1
k Zi > ¯ k + (¯ k)1/2 or i=1k Zi < ¯ k - (¯ k)1/2
• By independence of the Zi | ¿ , equivalent to C having “noisy” independent copies of the Zi
A Key Lemma• Lemma: For any protocol ¦ which succeeds w.pr. >.99, the
transcript ¿ is such that w.pr. > 1/2, for at least k/2 different i, H(Zi | ¿) < H(.01 ¯)
• Proof: Suppose ¿ does not satisfy this– With large probability,
¯ k - O(¯ k)1/2 i=1k Zi | ¿] < ¯ k + O(¯ k)1/2
– Since the Zi are independent given ¿, i=1
k Zi | ¿ is a sum of independent Bernoullis
– Since most H(Zi | ¿) are large, by anti-concentration, both events occur with constant probability:
i=1k Zi | ¿ > ¯ k + (¯ k)1/2 , i=1
k Zi | ¿ < ¯ k - (¯ k)1/2
So ¦ can’t succeed with large probability
Composition IdeaC
P1 P2 P3 Pk…
Z3Z2Z1Zk
The input to Pi in k-GAP-THRESH, denoted Zi, is the output of a 2-party Disjointness (DISJ) instance between C and Si
- Let S be a random set of size 1/(4ε2) from {1, 2, …, 1/ε2}- For each i, if Zi = 1, then choose Ti of size 1/(4ε2) so that DISJ(S, Ti) = 1, else choose Ti so that DISJ(S, Ti) = 0- Distributional complexity of solving DISJ with probability 1-¯/100, when DISJ(S,T) = 1 with probability ¯, is (1/ε2) [R]
DISJ
DISJ
DISJDISJ
Putting it All Together• Key Lemma ! For most i, H(Zi | ¿) < H(.01¯)
• Since H(Zi) = H(¯) for all i, for most i protocol ¦ solves DISJ(X, Yi) with probability ¸ 1- ¯/100
• For most i, the communication between C and Pi is (ε-2) – Otherwise, C could simulate the other players without any
communication and contradict lower bound for DISJ(X, Y i)
• Total communication is (k¢ε-2)
• Can show a reduction to estimating |x|0
Reduction to |x|0
• Think of C as a player• C’s input vector xC is characteristic vector
of the set [1/ε2] \ S
• Pi’s input vector xi is characteristic vector of the set Ti
• When |Ti Å S| = 1, support of x = xC + i xi usually increases by 1
• Choose ¯ = £(1/(ε2 k)) so thati=1
k Zi = ¯ k +- (¯ k)1/2 = 1/ε2 +- 1/ε
Talk Outline
• Lower Bounds– |x|0
– |x|2
Lower Bound for Euclidean Norm
• Improve (k + ε-) bound to optimal (k¢ε-2)
• Use Gap-Orthogonality (GAP-ORT(X, Y))– GAP-ORT(X,Y) = 1– Alice, Bob have X, Y 2 {0,1}ε-2
– Decide: |¢(X, Y) – 1/(2ε2)| <1/ε or |¢(X, Y) - 1/(2ε2)| >2/ε – Consider uniform distribution on X,Y
• [KLLRX, CKW] For any protocol ¦ that solves GAP-ORT with constant probability,
I(X, Y; ¦) = H(X,Y) – H(X,Y | ¦) = (1/ε2)
Information Implications
• By chain rule,
I(X, Y ; ¦) = i=11/ε2 I(Xi, Yi ; ¦ | X< i, Y< i) = (ε-2)
• For most i, I(Xi, Yi ; ¦ | X< i, Y< i) = (1)
XOR DISJ
• Choose random j 2 [n] and random S 2 {00, 10, 01, 11}:S = 00: j doesn’t occur in any Ti
S = 10: j occurs only in T1, …, Tk/2
S = 01: j occurs only in Tk/, …, Tk
S = 11: j occurs in T1, …, Tk
• Every j’ j occurs in at most one set Ti
• Output equals 1 if S 2 {10, 01}, otherwise output is 0• I(¦ ; T1, …, Tk | j, S, D) = (k) for any ¦ for which I(¦ ; S) = (1)
P1 Pk/2+1… PkPk/2
T1 Tk/2+1 Tk/2 Tk µ [n]
We compose GAP-ORT with a variant of k-Party DISJ
…
… …
GAP-ORT + XOR DISJ• Take 1/ε2 independent copies of XOR DISJ
– Ti = (Ti1, …, Ti
k), ji, Si, Di are variables for i-th instance• Is the number of outputs equal to 1 about 1/(2ε2) +-1/ε or
about 1/(2ε2) +- 2/ε?
XOR DISJ instance
XOR DISJ instance
…
XOR DISJ instance{1/ε2
Intuitive Proof• GAP-ORT is “embedded” inside of GAP-ORT + XOR DISJ
Output is XOR of bits in S
• Implies for any correct protocol ¦:For most i, I(Si ; ¦ | S< i) = (1)
• Implies via a direct sum:
For most i, I(¦ ; Ti | j, S, D, T< i ) = (k)
• Implies via the chain rule:
I(¦; T1, …, T1/ε2 | j, S, D) = (k/ε2)
• Implies communication is (k/ε2)
Conclusions• Tight communication lower bounds for estimating |
x|0 and |x|2
• Techniques imply tight lower bounds for empirical entropy, heavy hitters, quantiles
• Other results:– Model in which the xi undergo poly(n) additive updates
to their coordinates– Coordinator continually maintains (1+ε)-approximation
– Improve k2/poly(ε) to k/poly(ε) communication for |x|2